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ABSTRACT 



Determining the reliability of newly designed systems is one of the most important 
functions of the acquisition process in the military. Tracking the growth in reliability 
of a system as it is developed and modified repeatedly is an important part of the ac- 
quisition process. 

This thesis extends and expands a reliability growth simulation program written 
previously. It analyzes the capabilities and limitations of two discrete reliability growth 
models to determine which models are most applicable in estimating system reliability 
under a variety of different growth patterns. Negative growth patterns are also consid- 
ered. The result of this thesis is a FORTRAN simulation which enables a more accurate 
estimate of system reliability using test data generated during the development phase of 
an acquisition program. 
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THESIS DISCLAIMER 
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The reader is cautioned that computer programs developed in this research may not 
have been exercised for all cases of interest. While every effort has been made, within the 
time available, to ensure that the programs are free of computational and logic errors, 
they cannot be considered validated. Any application of these programs without addi- 
tional verification is at the risk of the user. 
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I. INTRODUCTION 



The reliability of a particular system or piece of equipment is one of the most im- 
portant factors in assessing its true value. A system with tremendous capabilities that 
spends an inordinate amount of time in the maintenance facility is usually less desirable 
than an alternate system with relatively moderate capabilities that has a much higher 
availability. Therefore, accurately measuring the reliability of a proposed system during 
the test and design phases is an integral part of the materiel acquisition process. Tradi- 
tional reliability estimators rely upon conducting a sulTicient number of tests or exper- 
iments so that a statistically valid point estimate or confidence interval can be 
established. These traditional estimates assume a constant or fixed reliability during any 
particular design phase. Since reliability will almost certainly fluctuate from phase to 
phase of the acquisition process, methods are needed that track reliability as it changes. 
Additionally, systems being produced presently are more complex and claim a much 
higher reliability than those produced just a few years ago. Verifying high reliability in- 
herently requires significantly more testing than does verifying relatively moderate reli- 
ability since items are tested until failure. Therefore, all test data should be utilized to 
the utmost. 

The process of testing and evaluating newly designed pieces of equipment is very 
costly in terms of both time and money. Typically, the test engineer must face limitations 
in both of these resources and occasionally must compromise on either the extensiveness 
of the test design or the number of trials to be run or both. It may be assumed that the 
constraints on time and money will not diminish in the near future — if anything, these 
constraints will become tighter. Therefore, methods that are able to produce accurate 
estimates of system reliability for a smaller investment of resources are very desirable. 
Reliability grovnh models are one such method. 

Reliability growth models make use of all available test data. Results from previous 
phases of design are combined with current test data so that a pattern of reliability is 
established. These models often make use of a procedure known as failure discounting. 
Failure discounting involves removing fractions of previous failures in order to make 
allowance for reliability improvement as the system evolves. The reliability pattern es- 
tablished by a reliability growth model becomes the basis for producing estimates of the 
actual system reliability. 
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There are two types of reliability growth models -- discrete and continuous. A 
continuous model is based on the time until failure of the system under consideration. 
Obviously, a number of distributional assumptions are necessary in a model of this type. 
Discrete reliability growth models are employed when test data references the number 
of system tests and failures in a particular design phase. In order to use a discrete reli- 
ability growth model one must be able to classify a test result as either a failure or a 
success. In other words, the test data must be attribute data. In this thesis, situations 
where test parameters may classify a trial as a partial success or a partial failure are not 
allowed. 

This thesis addresses discrete reliability growth models. The major objective is to 
analyze the capabilities and limitations of two such models. Two different methods of 
discounting previous failures are also evaluated. A previously designed simulation pro- 
gram [Ref. 1: pp. 36-46] is expanded or modified as necessary' in order to more fully de- 
velop the two discrete reliability growth models being examined. The original simulation 
uses actual reliabilities and Monte Carlo techniques in order to generate a random reli- 
ability growth pattern. The test data generated by this growth pattern is then used in the 
reliability growth models in order to produce estimates of system reliability. A variety 
of different reliability growth patterns and a broad spectrum of failure discounting pa- 
rameters are systematically evaluated. At each phase, the estimate of system reliability 
is compared to the actual reliability. The applicability of the two discrete reliability 
growth models, to include limitations and capabilities is also addressed. 

The following chapter describes each of the two discrete reliability growth models 
being considered. An explanation of the two failure discounting methods is also offered. 
Additionally, work done in this area previous to this analysis is summarized. Chapter III 
discusses the methodology used to conduct the analysis, to include the computer simu- 
lation. Chapter III also describes the various different growth patterns that were ana- 
lyzed. Chapter IV presents the results of the analysis. Chapter V details conclusions and 
recommendations. 
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II. DISCOUNTING PROCEDURES AND DISCRETE GROWTH 

MODELS 



A. BACKGROUND 

In this chapter, two discrete reliability growth models are described. Two separate 
techniques of discounting failures are also explained. Both methods remove a fraction 
of a previous failure when the system successfully completes a trial without a reoccur- 
rence of the particular failure cause. Reliability is defined as the probability of the sys- 
tem successfully completing a single trial. The exact definition of a test trial is much 
more difficult to fLx since, typically, the constitution of a trial is the responsibility of the 
testing agency. Normally, a trial may be considered as the exercising of the particular 
system in a manner consistent with its purpose. If a weapons system is being evaluated, 
for example, then a trial could consist merely of a single attempt at target acquisition 
or proceed through target acquisition to an attempt at target destruction. The precise 
definition of a trial depends in large measure on the purpose or goal of the test. For the 
purpose of this thesis and in order to utilize discrete reliability growth models, a trial 
must be defined so that it is discrete and can be evaluated as either a success or failure. 
Partial successes are not allowed. 

A testing phase may be defined as a number of trials, one or more, during which 
the configuration of the system is unchanged. Therefore, during a testing phase the ac- 
tual system reliability remains unchanged. A test phase may consist of one or more 
system failures. If improvements or changes to the system are effected after each system 
failure (a test-fix-test scenario) then a phase would consist of one system failure and all 
of the successful trials leading up to that failure. Alternatively, the test design may dic- 
tate that testing be continued until a certain number of failures occur. Under this sce- 
nario (test-find-test) the cause of system failure is identified but system configuration 
remains unchanged until the pre-designated failure occurs. In general, a phase will en- 
compass all of the trials between system configuration changes. 

B. FAILURE DISCOUNTING 

Testing conducted during the initial design stages of a particular system often indi- 
cates low reliability. Generally, weaknesses in the configuration of the system or defects 
in the quality of its components cause system failure. Test designs are established so 
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that the cause for these failures can be identified and corrected. Theoretically, then, as 
a weakness or a defect is identified and, hopefully, corrected the probability of that 
particular weakness or defect reoccurring should be reduced. This reduction in the 
probability of occurrence of a certain failure cause leads to improved system reliability. 
This concept is fully utOized in failure discounting. 

In order to effectively discount previous failures it is critical that the cause of the 
failure be properly identified. The level of detail that one wishes to ascribe to this iden- 
tification process is dependent upon the type of system being evaluated and the purpose 
of the test. If a complex system is being evaluated then a failure cause may be failure 
of a certain component or sub-component. The precise element that caused system 
failure is not critical but the ability to assign a failure cause to each system failure is. 
Correctly determining failure cause is very difficult, particularly when dehing with com- 
plex systems. Therefore, it is conceivable that design changes do not improve system 
reliability -- in fact, these changes may even degrade reliability. To apply the failure 
discounting procedures described below, one must be able to assign a failure cause to 
every system failure even though it may be done erroneously. 

1. Standard or Straight Percent Failure Discounting 

The standard discounting method will reduce a previous failure by a fixed 
amount after a predesignated number of trials have been successfully conducted without 
a reoccurrence of a failure due to the same cause. This method has been detailed fully 
in previous work [Ref. 1: pp. 14-17] so only a brief description and an example of its 
application will be included here. 

To employ the standard discounting method one must specify two parameters 
- the fraction of a failure to be removed, F, and the discount interval or the number of 
successful trials, I, that must occur before application of the discount. For the purpose 
of this paper and to use the simulation described herein, these parameters are specified 
at the outset and remain constant throughout the test. The number of successful trials 
since the occurrence of the failure is referred to as T. These three values, F, I, and T 
are then used to discount previous failures at the end of each phase. 

.Mathematically, the standard discounting method is given by Equation 2.1: 

DISCOUNTED FAILURE = (1 - (2.1) 

The term I NT in the above equation refers to the integer portion of the argument 
(“) • As such, if the number of successful trials, since the failure cause last occurred. 
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T 

T, IS less than the number of trials required by the discount interval, I, then Il\'T{-y) 
will be zero thus causing no discounting to be applied. Additionally, should a failure 
cause reoccur then the value of T is returned to zero thus removing any previous dis- 
counting that may have been applied. This ability, in effect, acts as a penalty factor. 
If a failure cause should reoccur then previous design changes may not have been ef- 
fective and thus any prior failure discounting could be* unwarranted. The rate and 
amount of failure discounting can be controlled by altering F and I. If the system under 
evaluation is relatively simple and a proven failure identification process is in place, then 
a large discount to system failures may be warranted. This will result in a prediction of 
rapid reliability growih. 



Table I. STANDARD DISCOUNT METHOD EXAMPLE 



PHASE 


TRIAL 


TOTAL 


CAUSE 


FAILURES 


X 


Y 


1 


1 


0 


S 


S 


1 


2 


I 


F 


S 


2 


3 


I 


S 


S 


2 


4 


1 


S 


S 


2 


5 


2 


S 


F 


3 


6 


2 


S 


S 


3 


7 


2 


S 


S 


3 


8 


2 


S 


S 


3 


9 


2 


S 


S 


3 


10 


2 


S 


S 


3 


11 


2 


S 


S 


3 


12 


3 


F 


S 



A small example of the application of the standard discount method will aid in 
clarifying its abilities. Consider the data in Table 1. This test data is purely fictitious 
and is intended solely to illustrate the use of the standard discount method. We will 
assume that improvements or repairs are effected after each failure so actual system re- 
liability is not constant. 

Table 1 represents three phases of a test-fix- test scenario where two failure 
causes, X and Y, have been identified. It should be noted that under a test-fix-test plan. 
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a phase will terminate with system failure. The result of each trial is listed as either a 
success or failure for a particular failure cause. 

The two parameters necessary to employ the standard discount method are the 
fraction of failure to be removed, F, and the discount interval, I. Assume that F = 0.50 
and I = 3. System failure number two, attributed to failure cause Y, terminates phase 
two. At that point the failure due to cause X that terminated the first phase has had 
three successive successful trials so failure discounting can be applied: 

ADJUSTED FAILURE=[\ - 0.50/f ^ = 0.50 

Thus, for reliability computations at the conclusion of the second phase, the failure that 
occurred during phase 1 is only counted as one-half of a failure. 

Phase three terminates with a failure due to cause X. The value for T then is 
set to zero and thus when applied to the failure that occurred in phase one, full value is 
restored. At the end of phase two this failure had been discounted to 0.50 but, because 
the failure cause reoccurred, this discount is viewed as unwarranted and the failure is 
returned to one. However, at the end of phase three failure cause Y has had seven suc- 
cessful trials. Therefore, discounting is applied: 

ADJUSTED FAILURE={\ - 0.50)^^^i^ = (0.50)^ = 0.25 

Therefore, when computing reliability estimates at the end of phase three, only 2.25 
system failures (failure number one plus failure number three plus the discounted value 
of failure number two) are considered instead of the three failures that actually occurred. 

The standard discount method described is very flexible because of its two pa- 
rameters. Setting values for F and I does, however, require a good deal of professional 
judgement and expertise. To avoid conflicts in setting such values David K. Lloyd pro- 
posed an alternate method of discounting failures [Ref. 2]. The Lloyd method is de- 
scribed in the next section. 

2. Lloyd Failure Discounting 

Discounting previous failures using the Lloyd discounting method is based on 
the premise that there should be some sort of statistical basis for determining how much 
a previous failure should be discounted. Lloyd offers the upper confidence limit of the 
probability that a failure cause will occur again as the discounted value of its respective 
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failure. The confidence limits for this probability are based on the number of successful 
trials since the failure cause last occurred. Under a test-fix-test scenario this is equivalent 
to the number of successful trials since the last failure. This number was defined in the 
description of the standard discount method as T. In order to implement Lloyd failure 
discounting, one must set the level of confidence desired for the confidence bound. Since 
the method is applied on every trial there is no requirement to specify an interval, al- 
though as will be explained in a later chapter, specifying an interval for the Lloyd 
method can be done and may be advantageous in some situations. 

The Lloyd discount method may be mathematically expressed as follows: 

ADJUSTED FAILURE = 1 - (1 - CPjT i/T>0 (2.2) 

In the above equation Cl is defined as the level of confidence desired and T is the 
number of successful trials. If T has a value of zero then the value of the adjusted failure 
is set to one. Setting the value of the adjusted failure equal to one tvhen T equals zero 
gives the Lloyd method the ability to restore all of the value of a discounted failure 
should its failure cause reoccur. 



Table 2. LLOYD DISCOUNT METHOD EXAMPLE 



PHASE 


TRIAL 


TOTAL 

FAILURES 


FAILURE 

CAUSE 


ADJUSTED 

FAILURES 


X 


Y 


I 


1 


0 


S 


S 


0 


1 


2 


1 


F 


S 


1.000 




3 


1 


S 


S 


0.900 


2 


4 


1 


S 


S 


0.684 


2 


5 


2 


S 


F 


0.536 + 1.000 = 1.536 


J 


6 




S 


S 


0.438 + 0.900 = 1.338 


J 


7 




S 


s 


0.369 + 0.684 = 1.053 


J 


8 


2 


S 


s 


0.319 + 0.536 = 0.855 


-» 

J 


9 


1 

Xm 


S 


s 


0.280 + 0.438 = 0.718 


J 


10 


2 


S 


s 


0.250 + 0.369 = 0.619 


J 


11 


1 

Xm 


S 


s 


0.226 + 0.319 = 0.545 


J 


12 


J 


F 


s 


2.000 + 0.280 = 2.280 
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Table 2 illustrates the use of the Lloyd method. The test data is the same data 
used to demonstrate the standard discount method. Since the Lloyd method is applied 
on every trial a fourth column has been added to the table reflecting the current value 
of the adjusted failure. Let us assume that the confidence interval desired in this case 
was .90. 

If reliability is computed at the completion of the third phase then 2.28 system 
failures would be used under the Lloyd discounting method as opposed to the actual 
value of three that occurred. The last column of the table indicates the speed that failures 
become discounted with the Lloyd method. 

The two discrete reliability growth models that will be examined are described 
in the next section. Each model makes use of discounted failure data and each has 
demonstrated, through simulation, the ability to estimate constant and concavely in- 
creasing reliabihty with reasonable accuracy. 

C. DISCRETE RELIABILITY GROWTH MODELS 

Both of the discrete reliability growth models to be presented assume a constant 
system reliability within a phase. The first model to be examined will be the iVIaximum 
Likelihood Estimate with Failure Discounting (MLEFD). Tliis model is merely a deriv- 
ative of the conventional single phase maximum likehliood estimate. The single phase 
maximum likelihood estimate considers only test data generated during the current 
phase. The MLEFD, conversely, is cumulative in that it considers all of the available 
test data. The test data from early phases during which the actual system reliability was 
different requires adjustment. Thus, the use of failure discounting techniques provides 
a method by which failures occurring early in the development process can be assim- 
ilated into current reliability estimates. The second model is a regression model based 
on an exponential single phase reliability estimate. It estimates reliability for each phase 
using an exponential model and then performs a linear regression on these estimates to 
obtain a current estimate. The simulation program that was written to evaluate these 
two models was designed so that failure discounting may or may not be invoked with 
either model. 

Before offering a general description of the two models it is important at this 
juncture to digress for a moment and discuss the exact methodology that will be used to 
incorporate discounted failures into the various reliability estimates. As was seen in the 
section prior, discounted or adjusted failures are rarely if ever integer-valued. While this 
does not present a problem for the MLEFD reliability growth model it does pose an 
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obstacle to using the exponential regression model. The convention devised in Captain 
Drake's thesis [Ref. 1: p. 23] will be continued in this paper. The number of trials up to 
and including a particular failure will be divided by the adjusted failure so that an ad- 
justed number of trials is computed. The adjusted failure value is then returned to one 
from its previous fractional value. The formula for computing adjusted trials is then; 



ADJUSTED TRIALS = 



TOTAL TRIALS 
ADJUSTED FAILURE 



(2.3) 



The appeal of this method is that the result is an integer number of failures while main- 
taining a constant ratio of number of failures to number of trials. While the adjusted 
number of trials may not be integer-valued they may be adjusted to integers, if necessary, 
through rounding with a much smaller effect on reliability computations than rounding 
adjusted failures. 

If a failure had been discounted (using either discounting method) from one to .25, 
for example, and the number of successful trials since the failure occurred was ten then 
the ratio of failures to successes would be .25 to ten. Using the method described above, 
the adjusted failure and actual trials are divided by the adjusted failure yielding a failure 
to success ratio of one to forty. Thus, the number of trials has been adjusted from ten 
to forty and the number of failures has been returned to one from .25. Since the ratio 
of failures to successes remains the same reliability computations will not be affected. 
Both of the models described below will make use of this method in computing reliability 
estimates. 

1. Maximum Likelihood Estimate with Failure Discounting 

The traditional estimate of system reliability is the maximum hkelihood esti- 
mate. Letting R denote the estimate of reliability, the maximum likelihood estimate may 
be expressed as follows: 

^ ^ TOTAL TRIALS - TOTAL FAILURES 
TOTAL TRIALS 



or alternatively: 

A ^ SUCCESSFUL TRIALS 
TOTAL TRIALS 



As stated previously, the maximum likelihood estimate assumes a constant re- 
liability between phases. It also requires a large number of trials to accurately estimate 
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the underlying system reliability. Failure discounting is used to make previous test data 
compatible with current results and thus increase the total number of trials available. 
However, since the actual system reliability is unknown it becomes a very non-trivial 
task to select the appropriate discounting method and the correct parameters. 

When previous test data is made available through the use of a discounting 
routine the equation for estimating the system reliability becomes: 

^ _ TOTAL TRIALS - TOTAL ADJUSTED FAILURES 

TOTAL TRIALS 

or if the convention of restoring the number of failures to integer values is adopted 

^ ^ TOTAL ADJUSTED TRIALS - TOTAL FAILURES ^ 

TOTAL ADJUSTED TRIALS 

To utilize the MLEFD reliability growth model one first records all of the test 
data from the current phase. The discounting method to be used is then performed on 
the previous test data and an adjusted failure value is computed for each previous failure. 
The adjusted number of trials for each failure is then computed using Equation 2.3. The 
. estimate of reliability for the current phase is then calculated using Equation 2.4 above. 

One deficiency of the maximum likelihood estimate is that, when testing is ter- 
minated after a fixed number of failures, the MLE is inherently a biased estimator of the 
actual reliability. The expected value of the maximum likelihood estimate has been de- 
rived in previous work [Ref 3: p.34]. It may be expressed mathematically as follows: 

EIR2 = \ + ( -^) ln(l -R)- ln(l - R) (2.5) 

A 

where R represents the actual reliability of the system. The bias, B{R), in the estimate 
may be calculated using the formula: 

B{R) = £[^] - R 

Table 3 below depicts the performance of the maximum likelihood estimate for 
different values of actual reliability, R. The bias is inherent to the estimate in that it will 
be present regardless of the validity of the assumption of constant phase reliability. 

The use of failure discounting can offset the effect of bias as well as handle 
changing system reliability. However, this puts a lot of additional weight on the ability 
to correctly select the best method and choose the proper parameters. Alternatively, if 
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an estimator of system reliability had to be biased one would want it to be conservatively 
biased as is the MLE. Underestimating actual system reliability is normally less costly 
than overestimating. However, a less biased estimate would be beneficial. 



Table 3. INHERENT BIAS IN THE MAXIMUM LIKELIHOOD ESTIMATE 



RELIABILITY 


1 1 


B(R) 


0.50 


0.307 


-0.193 


0.60 


0.389 


-0.211 


0.70 


0.484 


-0.216 


0.80 


0.598 


-0.202 


0.90 


0.744 


-0.156 


0.95 


0.842 


-0.108 


0.97 


0.892 


-0.078 


0.99 


0.953 


-0.037 



2. Exponential Regression Estimate 

The exponential regression reliability growth model was developed by H. 
Chernoff and W.M. Woods and based on the e.xponential single phase estimate [Ref. 
4]. The derivation of this model has been developed fully in previous work [Ref 5: pp. 
3-5] so for the purposes of this paper a general summary of results will be offered. 

The basic form of the exponential single phase estimate for a phase k i.e., after 
the kth improvement or alteration has been made, is stated in Equation 2.6; 

(2.6) 

where is the reliability estimate and the coefficient of the exponential term, Y, can 
be calculated as follows: 

n = l+y + y+...+ 

where is the number of trials up to and including the first failure in phase k. If X^ 
equals one then F* is defined equal to zero. It should be noted that tliis estimate was 
developed for the special case of testing until the first failure is encountered. If system 
failure occurs on the first trial {X^ =1) then F* will equal zero and system reliability will 
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be estimated as 1 - = 1-1 = 0. If failure occurs on the third trial, then the expo- 
nential single phase estimate would yield 1 — = 0.777. 

In order to evaluate the properties of the exponential single phase estimate a 
small simulation was designed. Geometric test data with actual reliability R were gener- 
ated using a transformation of a standard Uniform(0,l) random number. Ten thousand 
sets of geometric data were generated for each value of R. Table 4 represents the results 
of this simulation. Also included in the table is a simulation of the maximum likelihood 

A 

estimate, (R), using the same test data. 

A 

The values generated for the mean of the maximum likelihood estimate, R , 
conform fairly well with the theoretical results, thus the simulation itself is accurate. The 
values generated for the exponential estimate, R , while still underestimating the actual 
reliability, are clearly less conservatively biased than the maximum likelihood estimates. 
The sample variance and mean square error of each estimate is also represented in the 
table. The mean square error of each estimate is the sum of the variance and the square 
of the bias. It can be seen that as the actual reliability exceeded .70 the mean square 
error for the exponential estimate became less than the mean square error of the maxi- 
mum likelihood estimate. Therefore, the advantage of a lower variance for the maximum 
likelihood estimate at values of actual reliability below .90 is offset by the greater bias. 



Table 4. PROPERTIES OF THE EXPONENTIAL RELIABILITY ESTIMATE 



R 


1 — 1 


Maximum Likelihood Estimate 


Exponential Estimate 


A 

R 


Var{R) 


MSE{R) 


R 


Var{R) 


MSE{R) 


0.50 


0.307 


0.302 


O.IOl 


0.141 


0.357 


0.137 


0.157 


0.60 


0.389 


0.387 


0.111 


0.157 


0.451 


0.143 


0.166 


0.70 


0.484 


0.480 


0.115 


0.163 


0.547 


0.140 


0.163 


0.80 


0.598 


0.595 


0.107 


0.149 


0.661 


0.121 


0.140 


0.90 


0.744 


0.743 


0.079 


0.103 


0.797 


0.080 


0.090 


0.95 


0.842 


0.842 


0.050 


0.062 


0.881 


0.047 


0.052 


0.97 


0.892 


0.892 


0.034 


0.040 


0.921 


0.031 


0.033 


0.99 


0.953 


0.954 


0.014 


0.015 


0.968 


0.012 


0.012 
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The exponential regression reliability growth model developed by Chernoirand 
^Voods uses the technique of linear regression to estimate the coeH'icicnt of the expo- 
nential term. The model can be expressed mathematically as follows: 

Rk = 1 - ( 2 . 7 ) 

In the equation above k denotes the testing phase being used to compute the 
system reliability, a and ^ are computed at the end of each phase using the techniques 
of hnear regression thus yielding an estimate of system rehability. The exact derivation 
of the linear regression formula for x and /? is detailed in other sources [Ref 3: p. 3] and 
is provided at Appendix A for the interested reader. 

This model requires that both failures and trials be integers. Therefore, when 
adjusted trials are computed using Equation 2.3. the value obtained must be rounded to 
the nearest integer. Since linear regression is employed, this model has the ability to 
track changing reliability without the use of failure discounting. 

D. SUMMARY OF PREVIOUS WORK 

Analysis of reliability growth models has been a continuing process, and many 
advances have been made in determing their properties [Refs. 6.7]. It would require many 
pages to summarize all of the previous w'oi k in this area. For the purposes of this paper, 
the summary will detail only those results which have a direct bearing on the simulation 
being used to evaluate the tw'o reliability growth models. 

A simulation to evaluate these models was constructed by Captain James Drake 
[Ref 1: pp. 36-46]. This simulation w'as written to handle growth patterns where actual 
reliability was constant or growing at a fixed rate. Since one of the objectives of this 
paper is to expand the number of reliability growth patterns that can be simulated it is 
important, at this time, to summarize Captain Drake's work to include the assumptions 
necessary to run the simulation and some of the limitations. 

.Many reliability estimators are too mathematically complex to allow closed form 
solutions of their properties. In order to compare these properties computer simulation 
is one of the only techniques available. The simulation constructed by Captain Drake 
builds a reliability growth pattern and generates test data based on known system reli- 
abilities using the geometric distribution. The discrete reliability growtii model being 
evaluated has access to this test data for reliability estimation. The test data generated 
by the computer simulation is typically the number of trials up to and including failure 
and the cause of system failure for each predesignated failure in a phase. 
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The inputs required to run the original simulation are [Ref. 1: pp. 40-41]: 

1. Number of testing phases. 

2. Number of system failures allowed in each phase. 

3. Number of possible failure causes. 

4. Probability of non-occurrence of a particular failure cause in the first phase. 

5. Reliability growth fraction. 

6. Discounting option and parameters. 

Several assumptions were necessary to produce an adequate simulation based on 
these inputs. These assumptions are grouped into two categories — reliability growth 
pattern assumptions and failure cause assumptions. The assumptions required for simu- 
lating the reliability growth pattern are [Ref 1: pp. 37-38]: 

1. The reliability growth pattern is non-decreasing. 

2. System reliability changes only at phase boundaries. 

3. Equipment improvements are implemented immediately after a phase ends and 
before any further testing. 

4. Failure causes are corrected only at the end of a phase. 

5 . Each design improvement or repair removes a fixed fraction of the probability of 
reoccurrence for the corresponding failure cause. 

These assumptions narrowed the scope of the reliability growth patterns that could 
be evaluated. The first assumption does not allow the model to track any reliability 
growth patterns that exhibit periods of declining reliability. While for the most part ac- 
tual system reliabilities will be non-decreasing, this restriction does preclude a complete 
evaluation of the reliability estimators under consideration. Assumptions two and three 
are consistent with the previously stated definition of a testing phase. Assumption three 
does not allow modeling of long term design changes that while indicated, may be post- 
poned for whatever reason. Assumption five represents the method that is used in the 
simulation to model the impact of design changes or improvements. According to this 
assumption, any improvement or design change will remove a fixed fraction (user input 
number five) of the probability of occurrence for a failure cause. The underlying as- 
sumption is that all improvements are equally effective. 

These assumptions were necessary to generate the type of reliability growth patterns 
desired in Captain Drake's thesis. By introducing the concept of fixed phase reliability 
and modifying the original simulation accordingly, four of these five assumptions were 
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no longer necessary. The exact methodology used to effect this modification will be de- 
tailed in the next chapter. 

The last category of assumptions in the original simulation are the failure cause 
determination assumptions. These assumptions are [Ref 1 : p. 39]: 

1. There exists a finite number of possible failure causes. 

2. Each failure cause has a fixed probability of occurrence in each phase. 

3. System reliability can be modeled as a series system of the failure causes. 

4. Each failure cause is stochastically independent of the other failure causes. 

Assumption three dictates the method by which the actual system reliability is 
computed in the simulation. Since the occurrence of a failure cause, by definition, means 
system failure, modeling the system as a series is acceptable. This implies then that the 
actual system reliability may be mathematically expressed as: 



In the above equation ?,■ is the probability of occurrence of the ith failure cause and 
n is the total number of failure causes. 

The simulation is able to generate descriptive statistics about each of the reliability 
growth models being examined. These statistics include the mean, its confidence interval 
and the standard deviation for each reliability estimate at each phase. In order to 
produce these statistics the simulation should be replicated a sufficient number of times. 
The mean and the standard deviation are computed in the usual manner. The confidence 
interval calculation assumes a normal distribution. For the purposes of this paper, 500 
replications of each set of input data were conducted so the normality assumption 
should be reasonable. 

One of the limitations of the model in addition to the restricted number of reliability 
growth patterns that could be analyzed was in the application of the discounting meth- 
ods. The full range of input parameters could not be explored due, in large measure, to 
computer limitations. This limitation typically manifested itself at relatively high system 
reliability. If a failure cause with a very low probability of occurrence caused system 
failure early and if the input parameters for the discounting method were set such that 
a relatively large fraction of this failure was discounted over a relatively short number 



n 




( 2 . 8 ) 
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of successful trials then it was possible for this early failure to be discounted to essen- 
tially zero. When the adjusted trials were calculated utilizing Equation 2. 

{ADJUSTED TRIALS = ' ADfuSTEj^AtMw ^ approached infin- 

ity creating a system error in the computer. 

This problem was solved by specifying a lower bound for the adjusted failure. If the 
failure was discounted so that its adjusted value was less than .0000001 then the adjusted 
failure was set to .0000001. This modification allowed the full range of input parameters 
for the discounting methods to be evaluated. The effect on the estimate of system reli- 
ability is negligible. 
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III. METHODOLOGY 



One of the major objectives of this paper is to expand the simulation described in 
the previous chapter so that a greater variety of reliability growth patterns could be used 
to evaluate the two reliability growth models. This was accomplished through a modifi- 
cation of the simulation that allowed the user to input the desired system reliabilities at 
each phase. This chapter will detail the method that was used to effect this modification 
along with a description of the reliability growth patterns and failure discounting options 
that were used. 

A. FIXED PHASE RELIABILITY 

One of the limitations of the original simulation was that only constant reliability 
or reliability that increased at a constant rate could be modeled. If a predesignated fail- 
ure cause resulted in a failure then that failure cause's probability of occurrence on any 
succeeding phase was reduced by a predetermined constant amount. Thus, once the user 
of the simulation established the initial inputs for phase one, he had very limited control 
over the actual system reliabilities in the subsequent phases. 

In order to evaluate the two reliability growth models under a greater variety of 
potential reliability growth patterns the concept of fixed phase reliability was introduced 
as an option to the original simulation. With this modification of the program, the user 
has the ability to fix the actual system reliability at each phase. Thus, the reliability 
growth models can be used to estimate system reliability under any reliability growth 
pattern the user desires. 

Incorporating fixed phase reliability into the simulation necessitated an expansion 
of the user-provided input. The basic premise used in computing actual system reliability 
remains the same. This premise is that system reliability is the product of the probabili- 

n 

ties of success of the failure causes ( Equation 2.8, R = 0(1 ~ ^/) )• Originally, the 
simulation required that the user input the probability of success for each failure cause 
for the first phase. After that, the fractional improvement factor dictates the actual sys- 
tem reliability in succeeding phases. If the option of fixing the phase reliabilities is cho- 
sen, then the user must specify the probability of success for each failure cause in each 
phase. Equation 2.8 is then applied at each phase to yield the system reliability. 
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The table below illustrates the inputs that would be required to produce a growth 
pattern that initially declines and then increases rapidly. The scenario depicted is a sim- 
ple one involving five test phases with one failure per phase and two failure causes, X 
and Y. Desired reliability is user determined. 



Table 5. FIXED PHASE RELIABILITY EXAMPLE 



PHASE 


DESIRED 

RELIABIL- 

ITY 


USER INPUTS 


ACTUAL 

RELI- 

ABILITY 


PROBABILITY OF SUCCESS 


CAUSE X 


CAUSE Y 


1 


0.80 


0.90 


0.89 


0.801 


2 


0.70 


0.83 


0.84 


0.697 


3 


0.65 


0.77 


0.84 


0.647 


4 


0.75 


0.84 


0.89 


0.748 


5 


0.90 


0.94 


0.96 


0.902 



The actual reliability for each phase was computed using Equation 2.8. Consider 
phase one, for example. The desired system reliability is 0.80. To achieve this reliability, 
the probabilities of success for failure causes X and Y were input as 0.90 and 0.89 re- 
spectively. This yielded a computed system reliability of: 

n 

R = p|(l _ p^) = .90 X .89 = 0.801 

iMi 

Figure 1 is a graphical representation of the reliabilities computed in the table. The 
original simulation could not have produced this reliability pattern since declining reli- 
ability is present. This scenario may occur for systems during a portion of their devel- 
opment. While it is true that, in the long run, most systems exhibit non-decreasing 
reliabilities it is important to consider decreasing reliability when evaluating growth 
models. 

In order to invoke this ability to specify reliability at each phase, the number of user 
provided inputs increased from two to ten. If there had been two failures allowed in each 
of the five testing phases then the user inputs would have increased from ten to twenty. 
Therefore, while the option of allowing the user of the simulation to control the actual 
system reliability at each phase significantly increases the variety of different reliability 
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COMPUTED REUABILITY GROWTH PATTERN 




Figure 1. Fixed Phase Reliability Example 

growth patterns that can be analyzed, a rather large increase in the volume of input data 
is required. 

B. RELIABILITY GROWTH PATTERNS 

To more fully evaluate the properties and trends of the two reliability growth 
models described in the previous chapter, it was necessary to analyze their performance 
with respect to a variety of different reliability patterns. The eight growth patterns that 
were analyzed in this paper will be presented in this section. 

In conjunction with the reliability growth patterns, another important impact on 
the performance of the reliability estimators was the choice of discounting method. The 
table below depicts the various combinations of parameters for the standard discount 
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method that were analyzed. The discount fraction is the amount of a previous failure 
that is removed each time the discount method is applied while the discount interval is 
the number of successful trials between applications. The Lloyd discount method only 
has one parameter, the confidence interval. For the purposes of this paper, the Lloyd 
method was applied with confidence values of .8 and .9. 



Table 6. STANDARD DISCOUNT METHOD COMBINATIONS 



STANDARD DISCOUNT METHOD 


DISCOUNT FRACTION 


DISCOUNT INTERVAL 


0.00 


— 


0.25 


1 


0.25 


3 


0.25 


6 


0.25 


15 


0.50 


3 


0.50 


6 


0.50 


15 


0.75 


3 


0.75 


6 


0.75 


15 



The scenarios used to analyze the reliability growth models in this paper involved 
ten testing phases, i.e., testing was conducted until ten changes had been made. A total 
of five failure causes were considered in each test and a limit of one failure per phase 
was established. The simulation is able to handle any number of test phases, any num- 
ber of failure causes, and any number of failures per phase. The only limitation is the 
capacity of the computer on which the simulation is run. Another practical limitation 
involving the fixed reliability option is the volume of input data that would accompany 
a test design involving numerous phases or failure causes. As can be seen by the reli- 
ability growth patterns and user input tables following, the probabilities of success for 
the failure causes were allowed to fluctuate. The only requirement was that the reliability 
growth pattern established accurately reflected the desired growth pattern. 

Figure 2 and Table 7 describe the first reliability growth pattern. This is not a very 
conventional pattern in that most estimators assume concave growth patterns of the 
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form shown in Figures 5 and 6. However, this type of reliability growth pattern would 
not be unusual in situations where the exact method or technology reijuired to correct 
a lailure-causing defect is not immediately available but, as the system evolves and per- 
sonnel become more familiar with it, the the failure correction process proceeds more 
elTiciently, 

Figure 3 depicts a decreasing reliability growth pattern. In this instance, an at- 
tempted improvement actually caused system reliability to decrease. This situation, 
which is not eflectively addressed by any conventional reliability estimators, could pos- 
sibly occur in experimental or technologically advanced systems where the complete 
ramifications of design changes or corrections are not known. 

Figure 4 represents a scenario in which the system under consideration attains a 
moderately high reliability and then stagnates ai that level for several phases before 
starting to improve again. This scenario could occur if the e.xact cause of system failure 
is difficult to assess and would not be unconunon if the system being developed is highly 
complex. 

Figures 5 and 6 depict conventional reliability growth patterns one would expect to 
encounter when evaluating the majority of systems. In Figure 5, the system reliability 
increases rapidly to 0.99 and then remains constant whereas, in Figure 6, the system re- 
liability increases rapidly to 0.90 before becoming constant. It is important in the eval- 
uation of the reliability growth models to compare their performance in relatively 
atypical, although not uncommon, scenarios such as those shown in Figures 2 through 
5 with their performance in more conventional situations. 

Figures 7, 8, and 9 describe a constant system reliability of 0.90, 0.60, and O.40 re- 
spectively. While it would be rare to find these system reliabilities occurring in actual 
practice, their inclusion will illuminate some of the trends of the growth models under 
the various failure discounting methodologies. 

Each reliability growth pattern was simulated 500 times for each of the failure dis- 
counting combinations described previously. This will enable a more accurate appraisal 
of the capabilities and limitations of the two reliability growth models being considered. 
In addition, the most conmionly used estimator of reliability, the single phase maximum 
likelihood estimate, will be included on all figures so that a basis for comparing the two 
reliability growth estimators exists. The single phase maximum likelihood estimate, in 
contrast to the maximum likelihood estimate with failure discounting being evaluated in 
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PATTERN I 

RELIABILITY CONVEXLY INCREASING 




Figure 2. Pattern I 



Table 7. PATTERN I USER INPUTS 



CAUSE 


PHASE PROBABILITY OE SUCCESS 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


,S5 


.86 


.90 


.91 


.93 


.95 


.97 


.99 


.99 


.998 


0 


.84 


.85 


.87 


.90 


.92 


.95 


.97 


.99 


.99 


.998 




bo 


.84 


.86 


.88 


.90 


.93 


.96 


.98 


.99 


.998 


4 


.83 


.84 


.85 


.87 


.89 


.92 


.94 


.975 


.99 


.998 


5 


.81 


.83 


.84 


.86 


.89 


.91 


.94 


.961 


.99 


.998 



PArTERN II 

INCREASING - DECREASING - INCREASING 




Figure 3. Pattern II 



Table 8. PATTERN II USER INPUTS 



CAUSE 


PHASE PROBABILITY OF SUCCESS 


I 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


.98 


.98 


.99 


.99 


.99 


.99 


.99 


.99 


.99 


.9‘^ 


2 


.95 


.97 


.98 


.98 


.78 


.93 


.98 


.99 


.99 


.99 


3 


.82 


.93 


.96 


.72 


.90 


.95 


.97 


.98 


.98 


.98 


4 


.80 


.92 


.96 


.96 


.96 


.96 


.96 


.97 


.98 


.98 


5 


.66 


.85 


.90 


.90 


.90 


.90 


.94 


.96 


.96 


.96 
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PAHERN III 

INCREASING - CONSTANT - INCREASING 




Figure 4. Pattern III 



Table 9. PATTERN 111 USER INPUTS 



CAUSE 


PHASE PROBABILITY OF SUCCESS 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


.86 


.93 


.97 


.97 


.98 


.98 


.98 


.99 


.99 


.99 


2 


.86 


.93 


.97 


.97 


.98 


.98 


.98 


.99 


.99 


.99 


3 


.86 


.93 


.97 


.97 


.98 


.98 


.98 


.99 


.99 


.99 


4 


.86 


.93 


.97 


.97 


.98 


.98 


.98 


.99 


.99 


.99 


5 


.74 


.80 


.90 


.90 


.87 


.87 


.93 


.94 


.94 


.94 
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PAHERN IV 

RAPID INCREASE TO HIGH RELIABIUTf 




Figure 5. Pattern IV 



Table 10. PATTEPvN IV USER INPUTS 



CAUSE 


PHASE PROBABILI I Y OF SUCCESS 


1 


1 


3 


4 


5 


6 


7 


8 


9 


10 


1 


.98 


.99 


.99 


.995 


.998 


.998 


.998 


.998 


.998 


.998 


0 


.95 


.98 


.99 


.995 


.998 


.998 


.998 


.998 


.998 


.998 


3 


.82 


.96 


.99 


.995 


.998 


.998 


.998 


.998 


.998 


.998 


4 


.80 


.96 


.99 


.995 


.998 


.998 


.998 


.998 


.998 


.998 


5 


.6(3 


.90 


.99 


.995 


.998 


.998 


.998 


.998 


.998 


.998 
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PAHERN V 

RAPID INCREASE TO MODERATELY HIGH REUABIUTY 




Figure 6. Pattern V 



Table 11. PATTERN V USER INPUTS 



CAUSE 


PHASE PROBABILITY OF SUCCESS 


I 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


.ys 


.99 


.99 


.99 


.99 


.99 


.99 


.99 


.99 


.99 


2 


.95 


.98 


.99 


.99 


.99 


.99 


.99 


.99 


.99 


.99 


3 


.82 


.96 


.98 


.98 


.98 


.98 


.98 


.98 


.98 


.98 


4 


.80 


.96 


.97 


.975 


.975 


.975 


.975 


.975 


.975 


.975 


5 


.66 


.90 


.96 


.961 


.961 


.961 


.961 


.961 


.961 


.961 
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PAHERN VI 




CONSTANT MODERATELY HIGH RELIABIUTY 


q 




n 




d 




lO 








3 


- 






01 t 




d 




tN 




d 

o 


1 1 1 I 1 I 1 1 1 1 




2 + 5 a 10 




PHASE 



Figure 7. Pattern VI 



Table 12. PATTERN VI USER INPUTS 



CAUSE 


PHASE PROBABILITY OF SUCCESS 


I 


2 


3 


4 


5 


6 


n 

/ 


8 


9 


10 


I 


.99 


.98 


.98 


.975 


.97 


.97 


.975 


.98 


.98 


.99 


2 


.98 


.98 


.975 


.97 


.99 


.99 


.97 


.975 


.98 


.98 


3 


.98 


.975 


.97 


.99 


.98 


.98 


.99 


.97 


.975 


.98 


4 


.975 


.97 


.99 


.98 


.98 


.98 


.98 


.99 


.97 


.975 


5 


.97 


.99 


.98 


.98 


.975 


.975 


.98 


.98 


.99 


.97 
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PAHERN VII 

CONSTANT MODERATE RELIABILITY 
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Figure 8. Pattern VII 



Table 13. PATTERN VII USER INPUTS 



CAUSE 


PHASE PROBABILITY OF SUCCESS 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


.99 


.78 


.90 


.96 


.90 


.90 


.96 


.90 


.78 


.99 


2 


.78 


.90 


.96 


.90 


.99 


.99 


.90 


.96 


.90 


.78 


3 


.90 


.96 


.90 


.99 


.78 


.78 


.99 


.90 


.96 


.90 


4 


.96 


.90 


.99 


.78 


.90 


.90 


.78 


.99 


.90 


.96 


5 


.90 


.99 


.78 


.90 


.96 


.96 


.90 


.78 


.99 


.90 
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PATTERN VIII 



CONSTANT LOW RELIABIUTY 
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Figure 9. 



Pattern VIII 



Table 14. PATTERN VIII USER INPUTS 



CAUSE 


PHASE PROBABILITY OF SUCCESS 


1 


2 




4 


5 


6 


7 


8 


9 


10 


I 


.98 


.95 


.82 


.80 


.66 


.66 


.80 


.82 


.95 


.^8 


L, 


.95 


.82 


.80 


.66 


.98 


.98 


.66 


.80 


.82 


.95 




.82 


.80 


.66 


.98 


.95 


.95 


.98 


.66 


.80 


.82 


4 


.80 


.66 


.98 


.95 


.82 


.82 


.95 


.98 


.66 


.80 


5 


.66 


.98 


.95 


.82 


.80 


.8(.) 


.82 


.95 


.98 


.66 
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tilis paper, only considers the number of trials and failures in the current phase. All 
previous data is discarded since, presumably, the system reliability lluctuates from phase 
to phase thereby negating the underlying assumption of constant reliability. 

Of the eight reliability growth patterns being simulated, only four (including the 
three cases of constant system reliability, Figures 7, 8. and 9) could have been produced 
with the simulation as originally written. Therefore, the addition of fixed phase reliability 
will enable a more rigorous examination of the proposed reliability estimators. 
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IV. EVALUATION AND INTERPRETATION 



This chapter describes results of the simulation runs for each pattern. The per- 
formance of the models are reduced to graphical form to facilitate comparison. Only a 
few of the more representative runs are included in this chapter due to time and space 
constraints. The graphical results of many of the remaining simulation runs are in .Ap- 
pendix C . 

The analysis is organized according to the reliability growth patterns presented in 
the previous chapter. The performance of each of the two reliability growth models will 
be evaluated with respect to the growth pattern. The analysis focuses on how well the 
growth model tracked the actual reliability growth pattern. Included in this evaluation 
is an e.xamination of the standard deviations of the two estimators. 

It is not the purpose of this research to conduct enough simulation runs so that 
precise rules can be established for the selection of discounting parameters. This would 
require analyzing many more dilferent types of reliability growth patterns. In addition, 
the choice of discounting methods and their respective parameters is more a function of 
the particular system being evaluated than a function of the reliability growth model. 
Each system will have varying characteristics as will the personnel responsible for con- 
ducting the testing procedures. 

The reliability growth patterns will be referred to by their respective Roman nu- 
meral designations. These numerals are listed on the figures describing each pattern in 
the previous chapter and are summarized below for completeness: 

1. Pattern I - Convexly increasing reliability. 

2. Pattern II - Reliability initially increasing then decreases for several phases before 
resuming its upward trend. 

3. Pattern III - Reliability increases rapidly to approximately 0.80. Reliability then 
remains constant for several phases before increasing to its final level of approxi- 
mately 0.90. 

4. Pattern IV - Reliability increases rapidly to approximately 0.99. 

5. Pattern V - Reliability increases rapidly to approximately 0.90. 

6. Pattern VT - Reliability starts and remains constant at 0.90. 

7. Pattern VII - Reliability starts and remains constant at 0.60. 

8. Pattern VIII - Reliability starts and remains constant at 0.40. 
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A. CONSTANT RELIABILITIES - PATTERNS VI, VII, AND VTII 
1. Standard Discount Method 



PATTERN VI - NO DISCOUNTING 




Figure 10. Pattern VI- No Discounting 



Figure 10 shows the performance of the two reliability growth models with re- 
spect to Pattern VI. In this particular figure the discount fraction, F, is set at zero (pre- 
vious system failures are not discounted). y\s one might expect, both models performed 
well. Since the actual .system reliability is constant across all phases, the basic underlying 
assumption of constant reliability for the maximum likelihood estimate is valid. There- 
fore, previous data is compatible with current data without the use of failure discounting 
techniques. Thus, including all previous failure data such as done in the Maximum 
Likelihood Estimate with Failure Discounting (.VILEFD) will lead to a more accurate 
estimation of reliability. It is also interesting to note that the single phase maximum 
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likelihood estimate discussed brielly in the previous chapter resulted in an estimate of 
0.743 for phase ten. The single phase ma.xinium likelihood estimate considers only the 
number of trials and failures in the current phase and ignores past test data. I'he ex- 
pected value of this estimate, using Equation 2.5 is 0.744. thus supporting the validity 
of the simulation process. 



PATTERN VI - F = .25, I = 6 




Figure 11. Pattern VI, F = .25 and 1 = 6 

Both models did very' well in terms of the stability of the estimate when the 
actual system reliability was high. This is largely due to the fact that high system reli- 
ability implies that more test trials will be required to produce a failure. More test trials 
translates into additional data for the two models and, therefore, a less variable estimate. 
It should be noted, however, that when actual system reliability is constant, the MLEFD 
does better than the exponential regression model. The standard deviation of the 
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MLEFD is less than half that of the exponential regression estimate. Appendix C also 
contains the graphs of the .standard deviations of the two estimates for all eight patterns 
for the interested reader. 



PATTERN VIII - LLOYD, Cl = .9 




Figure 12. Pattern VIII, Lloyd method - Cl = .9 



The standard discount method will lead to an overestimation of reliability when 
applied to the MLEFD for the constant system reliability patterns. Figure II depicts 
the results of applying this method with the parameter values set at F = 0.25 and I = 
6. While only slightly alTeeting the exponential regression model, discounting previous 
failures in this manner resulted in the MLEFD overestimating system reliability fairly 
significantly after phase four. This is not unexpected since, as stated before, the as- 
sumption of constant system reliability is valid in this instance. It is important to note, 
however, the sensitivity of the MLEFD model to discount procedures compared to the 



34 



exponential regression model. Despite using a relatively small value for F the predicted 
reliabilities were markedly altered. This characteristic of the .\1LEFD is also evident in 
the patterns to follow. 

2. Lloyd Discount Method 

The Lloyd discount failure discounting method has only one parameter, the 
confidence interval. For the purposes of this study the only values for Cl that were ad- 
dressed were 0.80 and 0.90. The formula for applying the Lloyd discount method. 
Equation 2.2, is constructed so that the higher the value of Cl, the less a previous failure 
is discounted. Figure 12 graphically represents Pattern Vlll and the results of applying 
the Lloyd method with Cl = 0.90 to the two models. Once again, the efiect on the 
.MLEFD is rather dramatic with a phase ten estimate of reliability of approximately 0.70 
while the actual system reliability is 0.40. The effect on the exponential regression model, 
although significant after phase seven, is not nearly so severe. This marked elfect of 
applying the Lloyd discount method to the two reliability growth models continued with 
the other patterns as will be seen. Again, the important thing to note is the extreme 
sensitivity of the MLEFD to failure discounting in general and to the chosen values of 
the discount parameters in particular. 

B, RAPID RELIABILITY GROWTH - PAITERNS IV AND V 

The discussion of the results achieved for these patterns will center around Pattern 
V. Although the tendencies of the two models were the same for both patterns, the 
graphical portrayal of identified trends is better observed with this pattern than with 
Pattern IV where the final system reliability approaches 0.99. 

1. Standard Discount Method 

Figure 13 depicts Pattern V with the discount fraction, F, set at zero. Both the 
.MLEFD and the exponential regression model perform well. It is significant to note, 
however, that the exponential regression model converged to the actual system reliability 
much faster than did the MLEFD. This is due in large part to the use of linear regression 
techniques in this model. The standard deviation of the exponential regression was 
comparable to the .MLEFD for this pattern although still slightly larger. Both the 
.MLEFD and the exponential regression model outperformed the single phase maximum 
likelihood estimate thus lending validity to the use of reliability growth models to esti- 
mate system reliability. 

Figure 14 shows Pattern V with F = 0.50 and I = 3 trials. This high a rate of 
failure discounting caused the .MLEFD to yield a phase ten estimate of 0.997 when the 
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PATTERN V - NO DISCOUNTING 




Figure 13. Pattern V, No Discounting 

actual reliability was 0.900. A correlate of this high estimate of reliability is that the 
standard deviation associated with the .VILEFD became very small relatively quickly. 
The sensitivity of the MLEFD to failure discounting is again apparent in this figure. 

The exponential regression model, on the other hand, does remarkably well. It 
quickly approached actual system reliability. However, the standard deviation associated 
with this estimate remained high at a relatively constant value of 0.20. So, while the 
mean estimate of system reliability was very good, the associated variance of the esti- 
mate is greater than one would desire. 

The .\1LEFD performed quite well when F was set at 0,25 and I was put at 15 
as shown in Figure 15. The exponential regression model did slightly overestimate the 
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PAHERN V - F = .50, I = 3 




Figure 14. Pattern V, F = .5 and 1 = 3 

system reliability alter the sixth phase however, it was more proficient at matching the 
actual growth of reliability in the system. There was not a significant dilference in the 
standard deviations of the two estimates. 

2. Lloyd Discount Method. 

Figure 16 depicts Pattern V with a conlldencc interval value ofO.SO. Note that, 
as on the constant system reliability patterns discussed previously, application of the 
Lloyd discount method in its derived form caused both estimators to predict system re- 
liabilities that were markedly greater than the actual system reliability. This trend was 
consistent no matter what values of Cl were used. 
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PATTERN V - F = .25, 1=15 




Figuie 15. Pattern V, F = .25 and I = 15 

The conventional estimate of reliability, the single phase maximum likelihood 
estimate, is not affected by discount procedures since only current test data is consid- 
ered. The discounting methods addressed in this paper are only applied to failures oc- 
curring in previous phases. 

C. DECREASING RELIABILITY - PAITERN II 
1. Standard Discount Method 

Figure 17 shows Pattern 11 with no failure discounting applied. Of special in- 
terest in this graph is the performance of the two reliability growth models during the 
period of declining reliability. The exponential regression model was the only model that 
actually demonstrated a decrease in reliability although it still overestimated the actual 



38 



PAHERN V - LLOYD, Cl = .8 • 




Figure 16. Pattern V, Lloyd method, Cl = .8 

reliability during this period. The MLEFD, although slowing in terms of growth rate, 
failed to capture the trend of the actual reliability. The lack of ability to successfully 
predict the downward trend in actual reliability on the part of the .\ILEFD was present 
regardless of what values were assigned the discount parameters. In fact, as one might 
e.xpect, discounting previous failures e.xacerbated the problem. 

Figure 18 shows Pattern II with F = 0.75 and I = 6. The exponential re- 
gression model did very' well, particularly during the phases where system reliability was 
declining. The MLEFD, in contrast, performed poorly overestimating system reliability 
signilicantly after phase three and failing to indicate that system reliability decreased. 



39 



PATTERN II - NO DISCOUNTING 




Figure 17. Pattern II, No Discounting 



The standard deviation of the exponential regression estimate still exceeded that 
of the MLEFD. This characteristic is fairly consistent for all the patterns that were ex- 
amined in the course of evaluating these two models. It should be noted, however, that 
a small standard deviation does little good if the estimator is yielding markedly incorrect 
predictions. 

2 . Lloyd Discount Method 

Figure 19 shows Pattern II with Cl = 0.90. The application of the Lloyd 
method profoundly allected the performance of the MLEFD while only slightly altering 
the exponential regression model. This method of failure discounting did cause both 
models to overestimate system reliability by phase nine but, again, had less of an efl'ect 
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PATTERN II - F = .75. 1=5' 




Figure 18. Pattern II, F = .75, 1 = 6 

on the exponential regression model. It is especially noteworthy to observe the cIFect oC 
the Lloyd method on the MLEFD during the period of decreasing system reliability. The 
real sensitivity of the MLEFD to discount methods and parameters is partieularly evi- 
dent in this pattern. 

D. INTERMITTANT RELIABILITY GROWTH - PATTERN III 
I. Standard Discount Method 

Figure 20 depicts the performance of the two models when no failure dis- 
counting is applied. The exponential regression model outperformed the MLEFD at ev- 
er>' phase. .Again, the exponential regression model succeeded in capturing the trend of 
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PATTERN II - LLOYD. Cl = .9 




Figure 19. Pattern II, Lloyd method, Cl = .9 

the reliability growth pattern from phase four to phase siv whereas the MLEFD exhib- 
ited a constant rate of increase. 

When the discount parameters F and 1 were set at 0.25 and six respectively, tlie 
MLEFD did much better as can be seen in Figure 21. Both models converged to the 
correct prediction of system reliability by phase ten. However, the exponential regression 
model converged to the correct value much more quickly than did the MLEFD. The 
standard deviations of the two estimates were not too significantly dilTerent from each 
other for this particular pattern and discount method. 

The performance of the two models on reliability growth patterns 1 1 and 1 1 1 
demonstrate one of the inherent advantages of the exponential regression methodology. 
Since this method makes use of the techniques of linear regression, it is better able to 
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PAHERN III - NO DISCOUNTING , 




Figure 20. Pattern III, No Discounting 

track changing system reliability than is the MLEFD. This attribute of the exponential 
regression model will be developed further in the next chapter. 

2. Lloyd Discount Method 

Figure 22 shows how the models performed using Lloj'd failure discounting 
with Cl = 0.90. Again, applying the Lloyd method caused the MLEFD to overestimate 
system reliability after phase four. Some slight overestimation did occur in the expo- 
nential regression model during phases nine and ten however, the overall elfect was to 
enable the model to better track actual system reliability during the middle phases. 
During the discussion of the discounting methodologies in Chapter II, it was noted that 
the Lloyd failure discount method is applied after every trial that did not result in the 
reoccurrence of a previous failure cause. This means of application is a likely source of 
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PATTERN III - P = .25, I = 6 




Figure 21. Pattern III, F = .25, 1 = 6 

the overestiniation characteristics of both models when Lloyd discounting is applied. 
An idea on a possible remedy to this problem wall be presented at the end of this chapter. 

E. CONVEX RELIABILITY GROWTH - PATTERN I 

.Most reliability growth patterns are thought to be essentially concave in shape, 
much like Patterns IV and V. This is normally a valid assumption in that major im- 
provements in the reliability of a new system most often occur during the early stages 
of development as the obvious llaw's in the design and manufacture become apparent. 
It is not inconceivable, however, to experience a period of convex growth such as shown 
in Pattern I. One would want to have confidence in the chosen reliability estimator to 
correctly estimate system reliability regardless of its actual form. 
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PATTERN III - LLOYD, Cl - .9 . 




Figure 22. Pattern III, Lloyd method, Cl = .9 
1. Standard Discount Method 

Figure 23 shows the performance of the two models with no failure discounting 
being applied. It is apparent that, even without the benefit of failure discounting tecli- 
niques, the exponential regression model does well. It is able to reflect the trend of actual 
system reliability growth and the actual estimates are fairly close to the true system re- 
liability. The .MLEFD does not do so well and, in fact, is even outperformed by the 
single phase maximum likelihood estimate in the later phases. One would expect in this 
case to have significant results when failure discounting is applied due to the demon- 
strated sensitivity of the MLEFD to this procedure. 
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PATTERN I - NO DISCOUNTING 




Figure 23. Pattern I, No Discounting 

Figure 24 depicts Pattern I with F = .75 and I = 6. Although failing to capture 
the shape of the actual reliability pattern, the phase estimates produced by the MLEFD 
are quite good. Fractionally removing 75 percent of a previous failure after six successful 
trials is a substantial discount and one would expect that such a large discounting 
scheme would lend itself more to the conventional concave reliability growth patterns. 
This particular type of scenario would benefit greatly by the ability to alter the discount 
parameters during the course of the test. Initially, one would not want to discount pre- 
vious failures by a large amount since actual system reliability is only improving mar- 
ginally. As the system evolves and reliability begins to make substantial jumps then 
previous failures should be discounted more. Of course, this presumes a knowledge of 
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PAnERN I - F = .75, 1 = 6* 




Figure 24. Pattern 1, F = .75 and I = 6 



the actual system reliability in order to properly design such a plan and this is never the 
case. However, in depth knowledge of the proposed system could indicate the expected 
shape of the reliability pattern and this could be made use of in test design. .Although 
changing the discount parameters during the course of the test is possible in actual 
practice it is not, unfortunately, possible to do with the simulation in its current form. 
The parameters for the discount procedures remain constant once they are input at the 
beginning of the simulation. 

2. Lloyd Discount Method 

Figure 25 shows the Lloyd method with Cl = 0.90. The exponential regression 
model maintains the shape of the true system reliability while converging to the actual 
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PAHERN I - LLOYD. Cl = .9 




Figure 25. Pattern I, Lloyd method, Cl = .9 

reliability by phase eight. The MLEFD becomes concave in shape much as it did for the 
standard discount method shown in Figure 24. However, the intermediate phase reli- 
ability estimates are very good. The standard deviation of the MLEFD continues to be 
less than that of the expoiiential regression model although, as shown in Chapter II. the 
mean square error for the MLEFD exceeds that of the exponential regression model. 

The Lloyd method does better with this pattern than with any other. Previ- 
ously, application of this method caused both models to overestimate system reliability 
quite significantly. In this case, u.sing this method improved the performance ol' the ex- 
ponential regression model and, although distorting the shape, resulted in improved re- 
liability estimates from the MLEFD as well. 
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F. LLOYD FAILURE DISCOUNTING ( REVISITED ) 

Applying the Lloyd failure discounting method in its current form consistently 
caused both models to overestimate the true system reliability. This was true for the 
fixed phase reliability growth patterns that were addressed in this paper as well as for the 
growth patterns that were analyzed in Captain Drake's work previous to this [R.ef 1: pp. 
64-65]. One potential source of this problem is that the* Lloyd method is applied after 
every successful trial. 

The formula for computing adjusted failures using the Lloyd method is given by 
Equation 2.2 repeated below: 

ADJUSTED FAILURE = I - (1 - C/)^ ifT > 0. (2.2) 

If T equals zero then the adjusted failure value is set to one. This formula is used to 
compute adjusted failures after each trial in which the cause for a previous failure did 
not reoccur. Since the problem appears to lie in the number of times the method is ap- 
plied the inclusion of an interval may lead to more accurate results. 

A modified version of the Lloyd discount method which employs an interval of 
application is as follows: 

ADJUSTED FAILURE = 1 - (1 - C/)"v/ ifM > 0 (4.1) 

where, in this case. 



M = 7AT( 



T 



LDI 



LDl is defined as the Lloyd Discount Interval and becomes another parameter that can 
be altered by the user of the simulation. If .M equals zero then the adjusted failure value 
is set at one as before. By its construction, one can observe that for all values of T less 
than the predesignated LDl, then the value of M will be zero. M will increase by one for 
each group of successful trials meeting the LDl. Note also that specifying an LDI of one 
allows the user of the simulation to employ the Lloyd method in its original form. 

Some preliminary runs of the simulation were made with this modification included. 
The results were encouraging. Figure 12 e.Kamined previously illustrated the effect of the 
Lloyd method on the two models when system reliability was actually a constant 0.40. 
Figure 26 shows the result of applying this modilied version of the Lloyd method with 
the value for Cl remaining at 0.90 and the LDI set at eight. Both the .MLEFD and the 
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PATTERN VIII - LLOYD. Cl = .9, LDl = 5 



o 




Figure 26. Pattern VIII, Modified Lloyd Method, Cl = .9, LDI = 8 

exponential regression model performed much better than before. The LDI value of 
eight resulted in similar improvements for the other constant system reliability patterns. 

The results on the other patterns were similar to those represented in Figure 17 but 
the value for the LDI had to be adjusted for each pattern. Before any specific conclu- 
sions can be drawn about the benefits of this modification many more computer runs 
and more in depth analysis is required. However, based on these preliminary results, the 
inclusion of an interval does appear to have some merit. 
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V. SUMMARY, CONCLUSIONS AND RECOMMENDATIONS 



A. SUMMARY 

The purpose of tliis paper was to evaluate two discrete reliability grovvtli models. 
Each of these models has the ability to use fractionally discounted failure data to make 
an estimate of the actual system reliability at any point during the conduct of a test. 
Fractionally discounting failures occurring early in a test procedure allows the data ac- 
cumulated during the initial stages of a test to be used in current reliability estimates. 
Therefore, if the discounting procedure is performed wisely, more accurate reliability 
estimates with a higher degree of confidence should result. 

Two procedures for discounting previous system failures were described in Chapter 
II. The standard discount method requires the user to specify two parameters, the frac- 
tion of failure to be removed and the discount interval. The fraction of failure to be re- 
moved, referred to as the discount fraction or F, specifies how much a previous fiiilure 
is reduced upon each application of the discount procedure. The discount interval, I. is 
simply the number of successful trials (trials without a reoccurrence of the particular 
failure cause) that must occur between applications of the procedure. The other failure 
discounting method was proposed by David Lloyd and is referred to as the Lloyd dis- 
count method. This method utilizes only one input parameter, the level of confidence 
for an upper confidence interval on reliability. This method, in its original form, is ap- 
plied after every successful trial. A modified Lloyd discounting method which employs 
an interval of application was derived in Chapter IV'. 

Two discrete reliability growth models were described. The first model is a form of 
the maximum likelihood estimate with the addition of failure discounting. Since the 
maximum likelihood estimate assumes a constant reliability between phases, the addition 
of failure discounting will enable this model to estimate changing system reliability. The 
.Maximum Likelihood Estimate with Failure Discounting, or MLEFD, estimates system 
reliability as the ratio of the number of successful trials to the total number of trials with 
this ratio being computed after application of failure discounting. 

The other model was developed by H. Chernoff and VV.M. Woods and is a deriva- 
tion of an exponential single phase reliability estimate. This model employs linear re- 
gression techniques to estimate reliability after any change has been made to the unit 
of hardware under test. Because of this, the exponential regression model is able to track 
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changing system reliability without the benefit of failure discounting although the use 
of failure discounting can enhance the model’s performance. 

The means of evaluating these two models was a Monte Carlo simulation. The 
simulation used was originally written by Captain James Drake and was modified so that 
fixed phase reliabilities could be modeled. The fixed phase reliability modification allows 
the user of the simulation to evaluate the performance of the two reliability growth 
models on any pattern of actual reliability growth desired. Thus, periods of declining 
reliability, periods of no reliability growth, and convexly growing reliability among oth- 
ers can be easily modeled and the performance of the two growth models analyzed. 

Eight varying reliability growth patterns wxre analyzed. Both of the reliability 
growth models were evaluated against these eight patterns and under 13 diiferent failure 
discounting combinations including no failure discounting. Each model, pattern and 
failure discounting combination w^as replicated 500 times. The ability of each model to 
accurately track the actual system reliability wms evaluated along with an examination 
of the stability of the estimate produced by the model. 

B. CONCLUSIONS 

The evaluation of the two reliability growth models on the eight different reliability 
growth patterns leads to several general conclusions. The most obvious conclusion is 
that the use of either one of these grow'th models is superior to the standard single phase 
maximum likelihood estimate. Both growth models were far more accurate in terms of 
estimating the true system reliability and the variance associated wdth the model esti- 
mates was less than that produced by the single phase MLE. 

The MLEFD is particularly sensitive to the choice of failure discounting procedure 
and parameters. This sensitivity is made use of in order to allow the model to track 
changing reliability. However, this property also places a premium on the ability of the 
test engineer to correctly select the proper discounting method and parameters. 

The MLEFD did very well when the actual system reliability was constant. Con- 
stant system reliability is seldom present in actual reliability testing, however. The 
MLEFD also did w'ell when the true reliability was increasing at a constant rate. Al- 
though the exponential regression model more closely followed the actual reliability 
growth pattern, the MLEFD would be preferred in this instance because of its smaller 
variability. The .MLEFD did best with these patterns when the standard discount 
method was used with the discount fraction equal to 0.25 and the discount interval equal 
to 6 or 15. 
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The exponential regression model, although more variable in the early stages of 
testing, demonstrated the ability to accurately track each of the eight reliability growth 
patterns ev'aluated, even without the use of failure discounting. This model is also much 
less sensitive to the choice of discount method although its performance can be some- 
what enhanced through failure discounting. The MLEFD, on the other hand, lailed to 
track the decline in reliability in Pattern II. 

Therefore, the exponential regression model is recommended for situations where 
a proven means of failure discounting does not exist. If one is confident of the failure 
di.scounting methodology then the .MLEFD may be more advantageous due to the sta- 
bility of the estimates produced by it. Regardless of the degree of confidence one has in 
the failure discounting procedures, the exponential regression model is recommended if 
abnormal reliability grouih is anticipated. Abnormal reliability growth can be defined 
as a reliability pattern that has periods of decreasing or constant reliability in the middle 
of a generally increasing growth rate. The exponential regression model demonstrated 
superiority to the MLEFD in its ability to track dilFering patterns of growth. 

The Lloyd discount method, in its original form, caused both models to overesti- 
mate the actual system reliability. Therefore, use of this method with these two reliability 
growth models is not reconunended. A modified version of the Lloyd method which 
employs a discount interval demonstrated superior performance in some patterns to the 
original Lloyd discounting method. 

C. RECOMMENDATIONS FOR FURTHER STUDY 

The following are recommendations for further study and possible improvement to 
the models introduced in the study: 

• This study addressed only discrete reliability growth models. A similar study ana- 
lyzing the characteristics of continuous reliability growth models should be con- 
ducted. This would necessitate the construction of a dilFerent simulation. 

• Currently, the failure discounting mechanism used in the simulation employed in 
this analysis only allows the user to set parameter values at the beginnitig. T his 
should be modified so that discount parameters can be changed during the course 
of the simulation. 

• The failure discounting mechanism should be modified so that dilFerent parameters 
can be applied to different fiiilure causes. It may be desired that system failures 
caused by failure cause A, for example, be discounted less than those caused by 
failure cause B. This is not possible with the current simulation. 

• Further analysis is required of the modified Lloyd failure discounting method. 
Many more simulation runs with varying reliability growth patterns and dilFerent 
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combinations of discount parameters are required before any solid conclusions can 
be reached. 

• A weighted least squares method that weights recent data more heavily than pre- 
vious data should be analyzed to determine its ability to track reliability growth 
patterns with sudden changes in slope or direction. 
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APPENDIX A. DERIVATION OF THE EXPONENTIAL REGRESSION 

MODEL 



This model was developed by H. ChernolT and W.M. Woods. The derivation has 
been presented in previous works, most recently by W.M. Woods in a paper entitled 
"Reliability Growth Models" and by Captain James Drake in a thesis entitled "Discrete 
Reliability Growth Models using Failure Discounting". The exponential regression 
model is based on the exponential single phase reliability estimate. The exponential sin- 
gle phase estimate may be expressed matliematically as follows: 

R = \ - e~'* 

The exponential regression model uses linear regression to estimate A so 
A = a + pk 

where k is the testing phase being used to compute system reliability. Thus, the expo- 
nential regression model has the following formula: 



The estimates, a* and for a and /? at the end of the kth phase are obtained using 
the techniques of linear regression and an unbiased estimate for (a + pk) . Let F denote 
the total number of failures possible during the kth phase and let j equal the failure 
number in phase k such that j = 1,2,3.. ..,F . equals the number of trials between the 
(j-l)st failure and the jth failure, including the jth failure, in the kth phase. 

An unbiased estimate of (a + pk) using the jth set of trials in phase k is given by: 



If Sju = 1 (the first test was a failure) then is set equal to zero. The least squares 

A 

estimates, and P,,, for a and p at the kth phase are then: 



R 





if Nji, > 2 



k 



k 
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S, = r - 

where, 

= (^1/+ ^2/+... i- Ypi)IF 
K = {l + 2 + ... + k)lk 



Y = {Y, + Y, + ...+ Y,)lk 



Substituting these estimates for ct* and into the reliability equation yields the followin 
estimate of reliability for phase k: 

= 1 - + ^ > • 
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APPENDIX B. 












FORTRAN CODE FOR SIMULATION 






DISCxRETE RELIABILITY GROWTH SIMULATION 
PROGRAMMED BY JAMES E DRAKE 
MODIFIED BY JAMES D CHANDLER 
LAST MODIFIED 29 FEB 1988 



* THE FOLLOWING EXTERNAL FILES ARE USED BY THE PROGRAM 

* INPUT : DATA AND PARAMETER INPUT FILE (DEVICE # 10) 

* THESIS : OUTPUT FILE CONTAINING INTERMEDIATE COMPUTATIONS 

* (DEVICE // 20) 

* RELIAB: OUTPUT FILE CONTAINING FINAL RESULTS OF THE SIMULATION 

* (DEVICE ij 30) 

* EST ; OUTPUT FILE CONTAINING EACH PHASE ESTIMATE FOR EACH 

* REPLICATION OF THE WOODS WEIGHITD AVERAGE ESTIMATE 

* (DEVICE # 40) 

* MLEWD : OUTPUT FILE CONTAINING MLE ESTIMATES USING DISCOUNTING 

* FOR EACH PHASE AND EACH REPLICATION 

* (DEVICE # 50) 

* MLESP : OUTPUT FILE CONTAINING MLE ESTIMATE FOR EACH SINGLE PHASE 

* AND ALL REPLICATIONS USING NO DISCOUNTING 

* (DEVICE n 60) 

* REGEST : OUTPUT FILE CONTAINING EACH PHASE ESTIMATE FOR EACH 

* REPLICATION OF THE EXPONENTIAL REGRESSION ESTIMATE 

* (DEVICE # 70) 

* THE FOLLOWING IS A LIST OF KEY ARRAYS USED IN THE SIMULATION 



* 



A 



Vc 



* NFAPH 

* 

* NFCAUS 

* 



* NTRIAL 

■jV 

* PUREST 



MAIN WORKING ARRAY CONTAINS PROBABILITY OF SUCCESS FOR 
EACH FAILURE CAUSE, NUMBER OF TRIALS UNTIL FAILURE FOR 
EACH FAILURE CAUSE AND THE SYSTEM, CAUSE OF FAILURE, 
PHASE NUMBER, ADJUSTED NUMBER OF TRIALS AND ADJUSTED 
NUMBER OF FAILURES 

DIMENSION ( ((2*#CAUSES)+7), //FAILURES ) 

CONTAINS THE NUMBER OF FAILURES IN EACH PHASE 
DIMENSION (1, //PHASES) 

BINARY ARRAY USED TO DETERMINE IF A FAILURE OCCURRED IN 
A PHASE 

DIMENSION ( 1, //FAILURE CAUSES) 

CONTAINS THE NUMBER OF TRIALS SINCE LAST FAILURE OR 
DISCOUNTING FOR EACH FAILURE CAUSE 
DIMENSION ( 1, //FAILURE CAUSES ) 

RECORDS THE PHASE ESTIMATE FOR EACH ESTIMATOR WITHIN A 
SINGLE REPLICATION 
DIMENSION (4, //PHASES) 

ROW 1 ; WOODS WEIGHTED AVERAGE ESTIMATE 
ROW 2 : MLE WITH DISCOUNTING 



Vc 




ROW 3 . 


SINGLE PHASE MLE 


* 


Vc 




ROW 4 


EXPONENTIAL REGRESSION ESTIMATE 


•jV 


* 


AREL : 


CONTAINS ACTUAL SYSTEM RELIABILITY IN EACH PHASE 




* 




DIMENSION (1, /^PHASES) 


* 


* 


YJK : 


CONTAINS YJK VALUES UP TO 1000 




* 




DIMENSION (1,1000) 


* 


Vc 


CUMSF : 


CONTAINS THE NUMBER OF SUCCESS AND FAILURES FOR EACH 


■jV 


Vc 




FAILURE CAUSE (USED WITH WOODS WEIGHTED AVERAGE EST. ) 


Vr 


* 




DIMENSION (3, /^FAILURE CAUSES) 


* 


* 




ROW 1 


NUMBER OF FAILURES 




* 




ROW 2 


NUMBER OF SUCCESSES 


Vf 


Vc 




ROW 3 


ADJUSTED NUMBER OF SUCCESSES 


•5V 


Vc 


REG : 


ARRAY USED TO COMPUTE THE EXPONENTIAL REGRESSION ESTIMATE 


■)V 


* 




DIMENSION (5,#PHASES) 


•:V 


* 




ROW 1 


K BAR 


■>v 


* 




ROW 2 


Y BAR 


•;V 


Vc 




ROW 3 


Y BAR FOR THE PHASE 


•5V 


Vc 




ROW 4 


B HAT 


•5’C 


* 

“V 




ROW 5 


A HAT 


•5V 

•:V 


* 


THE REMAINING ARRAYS ARE USED TO COMPUTE THE MEAN AND VARIANCE 


Vr 


* 


OF EACH 


ESTIMATE AT EACH PHASE. THEY ALL HAVE THE SAME DIMENSIONS 




■jV 


AND STRUCTURE 




*:V 


* 




DIMENSION (4, //PHASES) 


•5V 


* 




ROW 1 


RUNNING SUM OF ESTIMATES 


* 


Vf 




ROW 2 


RUNNING SUM OF SQUARED ESTIMATES 


* 


* 




ROW 3 


MEAN OF THE ESTIMATES 


Vc 


* 

Vc 




ROW 4 


STANDARD DEVIATION OF THE ESTIMATES 


• 5 V 


* 


EST : 


VALUES 


FOR THE WOODS WEIGHTED AVERAGE ESTIMATE 


Vc 


* 


MLEWD : 


VALUES 


FOR THE MLE WITH DISCOUNTING 


* 


Vc 


MLESP : 


VALUES 


FOR THE SINGLE PHASE MLE 


■:V 


Vc 

* 


REGEST : 


VALUES 


FOR THE EXPONENTIAL REGRESSION ESTIMATE 


• 5 V 






C DEFINE AND DIMENSION VARIABLES 

PARAMETER (NR=50 ,NC=200) 

INTEGER REP,CUMSF,DISOPT,FRELOP,LDI,ALD 
REAL*4 MIN 

REAL* 8 DSEED , MLESP , MLEWD , EST , EUL 

DIMENSION NFAPH(NR) ,A(NR,NC) ,NFCAUS(NR) ,NTRIAL(NR) ,PHREST(4,NR) ,ES 
CT( 4 , NR) , MLEWD( 4 , NR) , MLESP( 4 ,NR) , REGEST( 4 , NR) , AREL( NR) , YJK( 1000 ) , (JU 
CMSF(3,NR),REG(5,NR) 

C READ IN THE NUMBER OF CAUSES TO BE USED ( NCAUSE ) AND THE NUMBER 
C OF PHASES ( NPHASE ) IN THE TEST 

READ(10,*) NCAUSE 
READ(10,*) NPHASE 

C CHECK IF FIXED RELIABILITY OPTION IS DESIRED. FIX EULER’S NUMBER. 
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oooooonon 



RE AD (10,*) FRELOP 
EUL = 0. 5772156648 

CREATE VARIABLES FOR THE ROW INDICES OF THE WORKING MATRIX ( A ) 

IPHASE: PHASE 

ISYSPR: ACTUAL COMPONENT RELIABILITY 

INTR: NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 

IFAILC: CAUSE OF THE FAILURE 

lADJF: ADJUSTED NUMBER OF FAILURES ED 

AFTER DISCOUNTING HAS BEEN APPLIED 
lADJT: ADJUSTED NUMBER OF TRIALS AFTER DISCOUNTING llAS BEEN APPLIED 
lYJK: YJK COMPUTED ON THE ADJUSTED NUMBER OF TRIALS 

IPHASE = (2*NCAUSE)+1 
ISYSPR = IPHASE +1 
INTR = ISYSPR + 1 
IFAILC = INTR + 1 
lADJF = IFAILC + 1 
lADJT = lADJF + 1 
I YJK = lADJT + 1 



C READ IN THE NUMBER OF FAILURES IN EACH PHASE ( NFAPH(I) ) AND 
C COMPUTE THE TOTAL NUMBER OF FAILURES IN THE TEST ( NFAIL ) 

NFAIL = 0 

DO 10 I=1,NPHASE 

READCIO,*) NFAPH(I) 

NFAIL = NFAIL + NFAPH(I) 

10 CONTINUE 



C INPUT THE PROBABILITY OF SUCCESS IN A SINGLE TRIAL FOR EACH CAUSE 
C IN EACH PHASE IF FRELOP EQUALS ONE. 

IF (FRELOP .EQ. 1) THEN 
DO 15 I=1,NCAUSE 
DO 15 J=l, NFAIL 
READ(10,*) A(I,J) 

15 CONTINUE 
ELSE 

C INPUT THE PROBABILITY OF SUCCESS IN A SINGLE TRIAL FOR EACH CAUSE 
C IN THE FIRST PHASE IF FRELOP EQUALS ZERO. 

DO 20 I=1,NCAUSE 

READ(10,*) A(I,1) 

20 CONTINUE 

ENDIF 

C INPUT THE REMAINING VARIABLES , THE NUMBER OF SUCCESSFUL TRIALS 
C BEFORE A DISCOUNT IS APPLIED (N); THE DISCOUNT FACTOR (R); THE SEED 
C FOR THE RANDOM NUMBER GENERATOR, GGUBFS, (DSEED); RELIABILITY 
C GROWTH FRACTION (FRIMP); TRIGGER FOR PRINTING INTERMEDIATE OUTPUT 
C (lOPT) 
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TRIGGERS FOR SAVING EACH ESTIMATE AT EACH PHASE FOR EACH ESTIMATOR 
lOPTl : WOODS WEIGHTED AVERAGE MODEL 
IOPT2 : MLE WITH DISCOUNTING 
IOPT3 : SINGLE PHASE MLE 
I0PT4 : EXPONENTIAL REGRESSION MODEL 
DISCOUNTING OPTION TRIGGER (DISOPT); LLOYD FAILURE DISCOUNTING 
PARAMETER (GAMMA); LLOra DISCOUNT INTERVAL 

READCIO,*) N 
READ(10,*) R 
READCIO,*) DSEED 
READCIO,*) FRIMP 
READCIO,*) NREP 
READCIO,*) lOPT 
READCIO,*) lOPTl 
READCIO,*) IOPT2 
READCIO,*) IOPT3 
READCIO,*) I0PT4 
READCIO,*) DISOPT 
READCIO,*) GAMA 
READCIO,*) LDI 

XNREP = NREP 
DSEEDl = DSEED 

C INITIALIZE THE ARRAYS USED TO COMPUTE THE MEAN AND STANDARD DEVIATION 
C OF EACH ESTIMATOR 

DO 30 J=1,NPHASE 
DO 30 1=1,4 

EST(I,J) = 0. 0 
MLEWD(I,J) = 0. 0 
MLESP(I,J) = 0. 0 
REGEST(I,J) = 0. 0 
PHREST(I,J) =0.0 

30 CONTINUE 

C COMPUTE AND STORE THE YJK VALUES UP TO 1000 

YJK(l) =0.0 
DO 40 1=1,999 

YJKCl+1) = YJK(I) + 1. O/I 
40 CONTINUE 

C COMPUTE AND STORE K BAR FOR THE EXPONENTIAL REGRESSION MODEL 

SUM = 0.0 
DO 50 I=1,NPHASE 
SUM = SUM + I 
REG(1,I) = SUM/ I 
50 CONTINUE 

C MAJOR REPETITION OF THE SIMULATION LOOP 
DO 500 REP=1,NREP 
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C INITIALIZE FAILURE CAUSE VECTOR (NFCAUS) AND (CUMSF) 
C COMPUTE THE INITIAL SYSTEM RELIABILITY 



REL = 1. 

DO 60 I=1,NCAUSE 
NFCAUS(I) = 0 
REL = REL * A(I,1) 

DO 60 J=l,3 

CUMSF(J,I) = 0 

60 CONTINUE 

C INITIALIZE COLUMN (FAILURE # ) COUNTER FOR THE WORKING ARRAY (A) 

J = 1 

C LOOP TO COMPUTE THE NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 
C AND THE CAUSE OF FAILURE FOR EACH FAILURE IN EACH PHASE 

DO 130 K=1,NPHASE 

C SKIP ACTUAL COMPONENT RELIABILITY COMPUTATION AFTER FIRST REP 
C AND FOR FIRST FAILURE 

IF(J. EQ. 1) GOTO 75 
IF(REP. GT. 1) GOTO 75 
REL = 1. 

C IF FIXED RELIABILITY OPTION IS SELECTED THEN PHASE RELIABILITIES 
C ARE COMPUTED AS FOLLOWS 

IF (FRELOP .EQ. 1) THEN 
DO 65 I=1,NCAUSE 
REL = REL*A(I,J) 

NFCAUS(I) = 0 
65 CONTINUE 

ELSE 

C COMPUTE NEW ACTUAL RELIABILITY FOR THE COMPONENT IN PHASE K 
DO 70 I=1,NCAUSE 

C INCREASE CAUSE PR( SUCCESS) IF IT CAUSED FAILURE IN THE PREVIOUS PHASE 
C COMPUTE NEXT PHASE RELIABILITY AND REINITIALIZE NFCAUS (NOT USED IF 
C FIXED PHASE RELIABILITY OPTION IS SELECTED). 

IF(NFCAUSd). EQ. 1) THEN 

A(I,J) = A(I,(J-D) + ((1. - A(I,(J-1)))*FRIMP) 
ELSEIF(NFCAUSd). NE. 1) THEN 

Add) = A(I,(J-1)) 

ELSE 

ENDIF 

REL = REL*Add) 

NFCAUS(I) = 0 
70 CONTINUE 
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END IF 



75 J1 = 1 

TRTOT = 0. 0 

C COMPUTE THE NUMBER OF TRIALS UP TO AND INCLUDING FAILURE AND THE 
C CAUSE OF FAILURE FOR EACH FAILURE IN THE PHASE 

DO 120 L=l,NFAPHfK) 

IF(REP. GT. 1) GOTO 90 
IF(J1. EQ. 1) GOTO 85 
IF (FRELOP .EQ. 1) GOTO 85 
DO 80 I=1,NCAUSE 
A(I,J) = A(I,(J-D) 

80 CONTINUE 

85 A(ISYSPR,J) = REL 

A(IPHASE,J) = K 

90 MIN = 7. 2E75 

DO 110 I=1,NCAUSE 

C ASSIGN # TRIALS FOR CAUSES WITH PR( SUCCESS) = 0 OR 1 

IF(A(I,J). GE. 1. ) THEN 
A((I+NCAUSE) ,J) = 7.2E75 
GOTO 100 

ELSEIF(A(I,J).EQ. 0. ) THEN 
A((I+NCAUSE),J) = 1. 

GOTO 100 
ELSE 
END IF 

C CONVERT UNIFORM (0,1) RANDOM VARIABLE TO GEOMETRIC (# TRIALS UNTIL 
C FAILURE ) FOR EACH FAILURE CAUSE. RECORD THE MIN # TRIALS FOR THE 
C CAUSES AS THE SYSTEM TRIALS UP TO AND INCLUDING FAILURE AND 
C RECORD THE FAILURE CAUSE 

A((I+NCAUSE) ,J) = INT(1. +(LOG(GGUBFS(DSEED))/LOG(A(I,J)))) 
100 IF(A((I+NCAUSE),J). LE.MIN) THEN 

MIN = A((I-l-NCAUSE),J) 

IMIN = I 
ELSE 
ENDIF 

110 CONTINUE 

A(IFAILC,J) = IMIN 
NFCAUS(IMIN) = 1 

C COMPUTE THE TOTAL OF TRIALS FOR THE MLE SINGLE PHASE ESTIMATE AND 
C INCREMENT FAILURE COUNTERS 

A(INTR,J) = MIN 

TRTOT = TRTOT + A(INTR,J) 

J = J + 1 
J1 = J1 + 1 
120 CONTINUE 
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C COMPUTE THE MLE ESTIMATE OF COMPONENT RELIABILITY FOR THIS PHASE AND 
C COMPUTE THE RUNNING SUM OF ESTIMATES AND THE SUM OF ESTIMATES SQUARED 
C FOR COMPUTATION OF THE MEAN AND STANDARD DEVIATION OF 'DIE ESTIMATE 

PHREST(3,K) = (TRTOT - NFAPH(K) )/TRTOT 
MLESP(1,K) = MLESP(1,K) + PHREST(3,K) 

MLESP(2,K) = MLESP(2,K) + ( PHREST( 3 ,K)**2) 

130 CONTINUE 

C INITIALIZE THE ADJUSTED NUMBER OF FAILURES TO 1 AND THE COUNT OF THE 
C NUMBER OF TRIALS SINCE FAILURE OR DISCOUNTING (NTRIALS(I) ) TO 0 
C IN PREPARATION FOR THE DISCOUNTING ROUTINE 

DO 140 J=1,NFAIL 

A(IADJF,J) = 1. 

140 CONTINUE 

DO 150 I=1,NCAUSE 
NTRIAL(I) = 0 
150 CONTINUE 

C DISCOUNTING ROUTINE REVIEWS ALL PAST FAILURES AND CAUSES TO DATE 
C AND DETERMINES IF THE DISCOUNTING CONDITIONS HAVE BEEN MET. COMPUTES 
C THE ADJUSTED FAILURES, THE ADJUSTED OF TRIALS AND YJK 

J = 0 

DO 300 K=1,NPHASE 
DO 200 L=1,NFAPH(K) 

J = J + 1 

C UPDATES THE NUMBER OF TRIALS SINCE FAILURE OR DISCOUNTING FOR EACH 
C FAILURE CAUSE 

ICAUSE = INT(A(IFAILC,J)+. 5) 

DO 160 I=1,NCAUSE 

IF( ICAUSE. EQ. I) THEN 
NTRIAL(I) = 0 
ELSEIFC ICAUSE. NE. I) THEN 

NTRIAL(I) = NTRIAL(I) + INT( A( INTO, J)+. 5) 

ELSE 
END IF 

160 CONTINUE 
200 CONTINUE 

C CHOOSE DISCOUNTING METHOD TO BE USED 

IF(DISOPT. NE. 2) GOTO 180 

C PERFORM LLOYD'S FAILURE DISCOUNTING METHOD 

DO 170 1=1, J 

II = INT(A(IFAILC,I)+. 5) 

IF(NTRIAL(I1).EQ. 0) THEN 
A(IADJF,I) = 1.0 
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GOTO 170 



ELSE 

ENDIF 

C THIS IS THE MODIFIED LLOYD METHOD USING A DISCOUNT INTERVAL. THE 
C ORIGINAL DISCOUNT METHOD MAY BE EMPLOYED BY SETTING LDI TO ONE. 

ALD = INT(NTRIAL(I1)/LDI) 

IF(ALD .EQ. 0) THEN 
A(IADJF,I) = 1. 0 
GO TO 170 
ELSE 

A(IADJF,I) = 1. 0 - ((1. -GAMA)**(1. 0/ALD)) 

ENDIF 

170 CONTINUE 
GOTO 210 

C PERFORMS STRAIGHT PERCENT FAILURE DISCOUNTING AND 
C COMPUTES THE ADJUSTED # OF FAILURES 

180 DO 190 1=1, J 

II = INT(A(IFAILC,I)+. 5) 

IF(NTRIAL(I1).EQ. 0) THEN 
A(IADJF,I) = 1. 

ELSEIF(NTRIAL(I1).GE.N) THEN 

A(IADJF,I) = A(IADJF,I)*((1. -R)**(NTRIAL(I1)/N)) 

ELSE 

ENDIF 

190 CONTINUE 

C ADJUSTS THE # TRIALS SINCE FAILURE OR DISCOUNTING FOR THOSE CAUSES 
C THAT HAVE MET OR SURPASSED THE DISCOUNTING THRESHOLD 
C FOR THE STRAIGHT PERCENT DISCOUNTING METHOD 

DO 205 I=1,NCAUSE 

IF(NTRIAL(I).GE.N) NTRIAL(I) = MOD(NTRIAL( I) ,N) 

205 CONTINUE 

210 TADJT = 0. 0 
TYJK = 0.0 
TPYJK = 0. 0 
K1 = 0 

DO 215 12=1,3 
DO 215 I=1,NCAUSE 
CUMSF(I2,I) = 0 
215 CONTINUE 

C COMPUTES THE ADJUSTED # OF TRIALS FROM THE ADJUSTED # OF FAILURES 
C AND COMPUTES THE SUM OF THE ADJUSTED # OF TRIALS FOR ESTIMATE COMP. 



PREL = 0. 0 
LTRIAL = 0 



C IF ADJUSTED FAILURES ARE APPROACHING 0 THEN ADJUSTED TRIALS MUST 
C BE PRE-SET. 
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DO 240 1=1, J 



IF(A(IADJF,I) .LE. .0000001) THEN 
A(1ADJF,1) = . 0000001 
ENDIF 

A(1ADJT,1) = A(1NTR,1)/A(1ADJF,1) 

TADJT = TADJT + A(1ADJT,1) 

C COMPUTE YJK FROM THE ADJUSTED OF TRIALS AND STORE THE SUM FOR 
C ESTIMATE COMPUTATION, USE ARRAY FOR # TRIALS < 1000 AND APPROX. FOR 
C VALUES > 1000 

N1 = N1NT(A(1ADJT,1)) 

1F(N1. LE. 1000) THEN 

A(1YJK,1) = YJK(Nl) 

ELSE1F(N1. GT. 1000) THEN 
X=N1 
Q=12*X 
T=X+1 
S=X+2 

A( lYJK , 1 )=( EUL+( LOG( X) )+(!/( 2*X) ) -( 1/ ( Q*T) )-(!/( Q*T*S ) ) ) 



ELSE 

ENDIF 

C DETERMINE IF A PHASE BOUNDARY HAS BEEN REACHED TO BEGIN ESTIMATE 
C COMPUTATION 

IFCl.EQ. 1) GOTO 225 

1F( A( IPHASE , 1) . NE. A( IPHASE , ( 1- 1) ) ) THEN 
C COMPUTE THE WOODS WEIGHTED AVERAGE ESTIMATE 



MAX = 0 
K1 = K1 + 1 

C DETERMINE THE FAILURE CAUSE WITH THE LARGEST # OF FAILURES 

DO 220 11=1,NCAUSE 

1F(CUMSF(1,11).GT. MAX) THEN 
MAX = CUMSF(1,11) 

ICOL = 11 
ELSE 
ENDIF 
220 CONTINUE 

C COMPUTE YJK VALUE FOR THE CURRENT PHASE ESTIMATE 

1F(CUMSF( 1, ICOL). LE. 1000) THEN 
AHATL = YJK(CUMSF(1,IC0L)) 

ELSE 1F( CUMSF( 1, ICOL). GT. 1000) THEN 
X = CUMSF(1,1C0L) 

Q=12*X 
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T=X+1 

S=X+2 

AHATL=( EUL+( LOG( X) )+(!/( 2*X) ) - ( l/( Q*T) )-(!/( Q*T*S ) ) ) 

ELSE 

ENDIF 

IX = CUMSF(1,IC0L) + CUMSF(3,IC0L) 

IF(IX. LE. 1000) THEN 

AHATU = YJK(IX) 

ELSEIFC IX. GT. 1000) THEN 
X = IX 
0=1 2*X 
T=X+1 
S=X+2 

AHATU=( EUL+( LOG( X) ) +( 1/ ( 2*X) ) - ( 1/ ( Q*T) ) - ( 1/ ( Q*T*S ) ) ) 

ELSE 

ENDIF 

C COMPUTE CURRENT PHASE RELIABILITY ESTIMATE 

AHAT = AHATU - AlIATL 
OREL = 1.0 - EXP(-AHAT) 

X = CUMSF(1,IC0L) + CUMSF(3,IC0L) 

C COMPUTE AND STORE THE WOODS WEIGHTED AVERAGE ESTIMATE 

PREL = ((LTRIAL*PREL)/X) + (( (X-LTRIAL)*CREL)/X) 

LTRIAL = CUMSF(1,IC0L) + CUMSF( 3 , ICOL) 

C COMPUTE THE PHASE AND GLOBAL AVERAGE FOR YJK USED IN THE EXPONENTIAL 
C REGRESSION ESTIMATES ARE 

REG(2,K1) = TYJK/(I-1) 

REG(3,K1) = TPYJK/NFAPH(K1) 

TPYJK = 0.0 

ENDIF 

C COMPUTE THE NUMBER OF FAILURES AND SUCCESSES FOR EACH FAILURE CAUSE 
C USED IN THE WOODS WEIGHTED AVERAGE ESTIMATE 

225 ICAUSE = INT(A(IFAILC,I)+. 5) 

DO 230 I1=1,NCAUSE 

CUMSF(2,I1) = CUMSF(2,I1) + INT( A( INTR , I) + .5) 

CUMSF(3,I1) = CUMSF(3,I1) H- N1 
230 CONTINUE 

CUMSFCl, ICAUSE) = CUMSF( 1 , ICAUSE) + 1 
CUMSF( 2, ICAUSE) = CUMSF( 2 , ICAUSE) - 1 
CUMSFC 3, ICAUSE) = CUMSF( 3 , ICAUSE) - 1 
TPYJK = TPYJK + A(IYJK,I) 

TYJK = TYJK + A(IYJK,I) 

240 CONTINUE 
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C REPEAT COMPUTATIONS FOR THE V/OODS WEIGHTED AVERAGE ESTIMATE FOR THE 
C FINAL PHASE 



MAX = 0 

K1 = K1 + 1 

DO 245 I1=1,NCAUSE 

IF(CUMSF( 1,’I1). GT. MAX) THEN 
MAX = CUMSF(1,I1) 

ICOL = II 
ELSE 
END IF 
245 CONTINUE 

IF( CUMSF( 1 , ICOL) . LE. 1000 ) THEN 
AHATL = YJK(CUMSF( l.ICOL)) 

ELSEIF(CUHSF(l,ICOL). GT. 1000) THEN 
X = CUMSF(l,ICOL) 

Q=12*X 

T=X+1 

S=X+2 

AHATL=( EUL+( LOG( X ) ) +( 1 / ( 2*X ) ) - ( 1/ ( Q*T ))-(!/( Q*T*S ) ) ) 

ELSE 

ENDIF 

IX = CUMSF(l,ICOL) + CUMSF(3,ICOL) 

IF(IX. LE. 1000) THEN 
AHATU = YJK(IX) 

ELSEIFdX. GT. 1000) THEN 
X = IX 
Q=12*X 
T=X+1 
S=X+2 



AHATU=( EUL+( LOG( X ) ) +( 1/ ( 2*X ) ) - ( 1 / ( Q*T) ) - ( 1 / ( Q*T*S ) ) ) 

ELSE 

ENDIF 

AHAT = AHATU - AHATL 
CREL = 1. 0 - EXP(-AHAT) 

X = CUMSF( 1 , ICOL) + CUMSF(3,IC0L) 



PREL = ((LTRIAL*PREL)/X) + ( ( (X-LTRIAL)*CREL)/X) 

LTRIAL = CUMSF( l,ICOL) + CUMSF( 3 , ICOL) 

REG(2,K1) = TYJK/CJ) 

REG(3,K1) = TPYJK/NFAPH(K1) 

PHREST(1,K) = PREL 

C COMPUTE THE MLE ESTIMATE OF PHASE RELIABILITY USING DISCOUNTING 
PHREST(2,K) = (TADJT - J)/TADJT 

C COMPUTE THE EXPONENTIAL REGRESSION ESTIMATE BEGINNING WITH B HAT 
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SUM = 0. 0 
SUMS = 0. 0 

IF (K. EQ. 1) GOTO 252 
DO 250 I = 1,K 

SUM = SUM + (d-REG( 1,K))*REG(3,I)) 

SUMS = SUMS + (d-REG(l,K))**2) 

250 CONTINUE 

REG(4,K) = SUM/SUMS 

C COMPUTE A HAT 

REG(5,K) = REG(2,K) - (REG(4,K)*REG(1,K)) 

C COMPUTE AND STORE THE EXPONENTIAL REGRESSION ESTIMATE 

PIIREST(4,K) = 1.0 - EXP(-(REG(5,K) + ( REG( 4,K)*K) ) ) 
IF(PHREST(4,K). LT. 0. 0) PHREST( 4 , K)=0. 0 
GOTO 255 

252 PHREST(4,K) = 1. 0 - EXP( -REG( 3 , 1) ) 

IF(PHREST(4,K). LT. 0. 0) PHREST(4,K)=0. 0 

C STORE THE RUNNING SUM OF THE ESTIMATES FOR THE CURRENT PHASE AND THE 
C RUNNING SUM OF THE ESTIMATES SQUARED FOR COMPUTATION OF THE MEAN AMD 
C STANDARD DEVIATION OF EACH ESTIMATE FOR EACH RELIABILITY GROWTH 
C MODEL 

255 EST(1,K) = EST(1,K) + PHREST(1,K) 

EST(2,K) = EST(2,K) + ( PHREST(1 , K)**2) 

MLEWD(1,K) = MLEWD(1,K) + PHREST(2,K) 

MLEWD(2,K) = MLEWD(2,K) + ( PHREST( 2 ,K)**2) 

REGEST(1,K) = REGEST(1,K) + PHREST(4,K) 

REGEST(2,K) = REGEST(2,K) + (PHREST(4,K)**2) 

C STORE THE ACTUAL PHASE RELIABILITY 

AREL(K) = A(ISYSPR,J) 

C PRINT INTERMEDIATE OUTPUT IF REQUESTED AND THE NUMBER OF REPETITIONS 
C IS NOT GREATER THAN 5 

IFdOPT. NE. 1) GOTO 300 
IF(REP.GT. 5) GOTO 300 

WRITE(20,1000) REP,K 

1000 F0RMAT(T16, 'REPETITION NUMBER: ',14,' PHASE NUMBER: ',14) 

WRITE(20,1010) A(ISYSPR,J) 

1010 F0RMAT(22X, 'ACTUAL COMPONENT RELIABILITY: ' ,F7. 5) 

WRITE(20,1020) PHREST(1,K) 

1020 F0RMAT(20X, 'PREDICTED COMPONENT RELIABILITY: ',F7.5) 
WRITE(20,1022'i PHREST(2,K) 

1022 FORMAT(20X, 'MLE ESTIMATE USING DISCOUNTING: ',F7.5) 

WRITE(20,1025) PHREST(3,K) 

1025 F0RMAT(18X, 'MLE ESTIMATE OF PHASE RELIABILITY: ' ,F7. 5) 
WRITE(20,1027) PHREST(4,K) 
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1027 FORMATC 14X, 'REGRESSION ESTIMATE OF PHASE RELIABILITY: ’,F7.5) 
WRITE(20,1030) 

1030 FORMATC ' ' ') 

DO 260 I=1,NCAUSE 

WRITE(20,1035)I,A(I.J) ,A( ( I+NCAUSE) ,J) 

1035 FORMATC 12X, 'CAUSE: \I3,' PR(SUCCESS): ',F7.6,' C/ TRIALS: ', 
CFIO. 0) 

260 CONTINUE 

WRITEC20, 1036) 

1036 FORMATC ' ' , ' ' ) 

WRITEC20,1040) 

1040 FORMATC 4X, 'FAIL #',3X,'FAIL CAUSE ', 3X, ' # TRIALS ', 3X, ' AD J FAIL', 3 

CX,'ADJ # TRIALS' ,7X, 'yJK' ) 

DO 270 1=1, J 

WRITEC20,1050)I,ACIFAILC,I) ,AC INTR, I) , AC lADJF, I) ,AC lADJT, I) ,AC lYJK 
C,I) 

1050 FORMATC 4X, 13, 8X, F3. 0, 7X, F8. 0,4X,F8. 6, 4X,F12. 0,3X,F11. 4) 

270 CONTINUE 

WRITEC20,1060) 

1060 FORMATC' ',///) 



300 CONTINUE 

C PRINT EACH OF THE 3 ESTIMATES TO THEIR APPROPRIATE OUTPUT FILE 
C IF REQUESTED 



IFCIOPTl.NE. 1) GOTO 401 

400 WRITEC40,2000) CPHRESTC 1, I) , 

401 IFCIOPT2.NE. 1) GOTO 402 
WRITEC50,2000) CPHRESTC2 , I) , 

402 IFCIOPT3.NE. 1) GOTO 403 
WRITEC60,2000) C PHRESTC 3 , I) , 

403 IFCI0PT4. NE. 1) GOTO 500 
WRITEC 70,2000) CPHRESTC4, I) , 

2000 FORMATC' ' , 30C F7. 6: IX) ) 



I=1,NPHASE) 

I=1,NPHASE) 

I=1,NPHASE) 

I=1,NPHASE) 



500 CONTINUE 



C UPON COMPLETION OF ALL REPETITIONS, COMPUTE THE MEAN AND STANDARD 
C DEVIATION OF EACH ESTIMATE FOR EACH PHASE SKIPPING COMPUTATIONS IF 
C ONLY ONE REPETITION IS REQUIRED 

IF CNREP. LE. 1) GOTO 601 

DO 600 I=1,NPHASE 
ESTC3,I) = ESTC1,I)/XNREP 
MLEWDC3,!) = MLEWDC 1,I)/XNREP 
MLESPC3,!) = MLESPC1,I)/XNREP 
REGESTC3,!) = REGESTC 1 , I ) /XNREP 

ESTC 4,1) = SQRTC C ESTC 2 , 1 ) -C XNREP*C ESTC 3 , 1 )**2 ) ) ) /C XNREP - 1 ) ) 

MLEWDC 4 , 1 ) = SQRTC C MLEWDC 2 , 1 ) -C XNREP*C MLEWDC 3 , 1 )**2 ) ) ) / C XNREP- 1 ) ) 
MLESPC 4,1) = SQRTC C MLESPC 2 , 1 ) - C XNREP*C MLESPC 3 , 1 ) **2 ) ) ) / C XNREP- 1 ) ) 
REGESTC 4,1) =SQRTC C REGESTC 2 , 1 ) - C XNREP*C REGESTC 3,1) **2 ) ) ) / C XNREP - 1 ) ) 
600 CONTINUE 

C PRINT THE FINAL OUTPUT TABLE TO A FILE 
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601 WRITEf30,3000) 

3000 FORMATC 'O' ,T47,' DISCRETE RELIABILITY GROWTH SIMULATION') 
WRITE(30,3010) 

3010 FORMATC ' -' ,T54, 'MODEL PARAMETER SUMMARY' ) 

WRITE(30,3020) NCAUSE 

3020 FORMATC 'O' ,T47, 'NUMBER OF POSSIBLE FAILURE CAUSES ',14) 

IF CFRELOP .EQ. 1) GOTO 4000 
WRITEC30,3030) 

3030 FORMATC 'O' ,T38, 'CAUSE NUMBER' ,T64 ,' SINGLE TRIAL PRC SUCCESS ) FOR 
CPHASE 1 ' ) 

DO 3050 M=l, NCAUSE 
WRITEC30,3040) M,ACM,1) 

3040 FORMATC' ' ,T43 , 12 ,T79 ,F8. 6) 

3050 CONTINUE 

WRITEC30,3060) FRIMP 

3060 FORMATC 'o' ,T37, 'FRACTION CAUSE RELIABILITY IMPROVES AFTER FAILURE 
C' ,F8. 6) 

5000 WRITEC 30,3080) NPHASE 

3080 FORMATC'-' ,T48, 'NUMBER OF PHASES IN THE SIMULATION ',12) 
WRITEC30,3090) 

3090 FORMATC 'O' ,T42, 'PHASE NUMBER ' ,T59 ,' NUMBER OF FAILURES IN THE FIRST 
C PHASE') 

DO 3110 M=l, NPHASE 
WRITEC30,3100) M,NFAPHCM) 

3100 FORMATC' ' ,T43 , 12 ,T73 , 12) 

3110 CONTINUE 

WRITEC 30,3120) NFAIL 

3120 FORMATC 'O' ,T51, 'TOTAL NUMBER OF FAILURES ',14) 

IFCDISOPT. EQ. 2) GO TO 3160 
WRITEC30,3130) 

3130 FORMATC '-' ,T38, 'DISCOUNTING PERFORMED USING THE CONSTANT FRACTION 
CMETHOD ' ) 

WRITEC 30, 3 140) R 

3140 FORMATC 'o' ,T44, 'FRACTION EACH FAILURE IS DISCOUNTED ' ,F8. 6) 

WRITEC 30, 3 150) N 

3150 FORMATC' ' ,T33 ,' NUMBER OF TRIALS AFTER A FAILURE BEFORE A DISCOUNT 
C IS APPLIED ' ,14) 

GO TO 3190 
3160 WRITEC30,3170) 

3170 FORMATC '-' ,T44, 'DISCOUNTING PERFORMED USING THE LLOYD METHOD') 
WRITEC 30, 3 180) GAMA 

3180 FORMATC 'O' ,T39, 'PERCENT C. I. C USED AS DISCOUNT FRACTION ) ' ,F8. 6 
C) 

WRITEC 30, 3 185) LDI 

3185 FORMATC 'O' ,T50,' LLOYD DISCOUNT INTERVAL: ',13) 

3190 WRITEC 30, 3200) DSEEDl 

3200 FORMATC '-' ,T4 6, 'RANDOM NUMBER SEED USED ' ,F15. 2) 

WRITEC 30, 32 10) NREP 

3210 FORMATC 'O' ,T3 7, 'NUMBER OF REPETITIONS OF THE SIMULATION PERFORMED 
C ,17) 

WRITEC30,3220) 

3220 FORMATC ' 1 ' ,T61 , ' ESTIMATOR: ' ) 

WRITEC 30, 3230) 

3230 FORMATC 'o' ,T48, 'SINGLE PHASE MLE WITHOUT DISCOUNTING') 

WRITEC 30, 3240) 

3240 FORMATC '-' ,T60, 'MEAN' ,T83, 'ESTIMATE' ,T109, '95 %' ) 
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WRITE(30,3250) 

3250 FORMAT( ' ',T12, 'PHASE NUMBER' ,T29 ACTUAL RELIABILITY' ,T52, ' PPEDIC 
CTED RELIABILITY' ,T7 8, ' STANDARD DEVIATION' ,T101 ,' CONFIDENCE INTERVA 
CL') 

C COMPUTE C. I. FOR SINGLE PHASE MLE 

DO 3270 M=1,NPHASE 

Cl = (1. 96*MLESP(4,M))/SQRT(XNREP) 

CIU = MLESP(3,M) + Cl 
CIL = MLESP(3,M) - Cl 

WRITE(30,3260) M,AREL(M) ,MLESP(3,M) ,MLESP(4,M) ,CIL,CIU 
3260 F0RMAT('0',T17,I2,T34,F8.6,T58,F8. 6,T82,F9.6,T99,'( '.F8.6,' , ' ,F 
C8.6,' )') 

3270 CONTINUE 

WRITE(30,3220) 

WRITE(30,3280) 

3280 FORMATC 'O' ,T42, 'MAX LIKELIHOOD ESTIMATE USING DISCOUNTED FAILURES' 
C) 

WRITE(30,3240) 

WRITE(30,3250) 

C COMPUTE C. I. FOR MLE WITH DISCOUNTING 

DO 3290 M=1,NPHASE 

Cl = (1. 96*MLEWD(4,M))/SQRT(XNREP) 

CIU = MLEWD(3,M) + Cl 
CIL = MLEWD(3,M) - Cl 

WRITE(30,3260) M,AREL(M) ,MLEWD(3,M) ,MLEWD(4,M) , CIL, CIU 
3290 CONTINUE 

WRITE(30,3220) 

WRITE(30,3300) 

3300 FORMATC 'O' ,T38,' WEIGHTED AVERAGE ESTIMATE USING FAILURE DISCOUNTIN 
CG') 

WRITE(30,3240) 

WRITE(30,3250) 

C COMPUTE C. I. FOR WOODS WEIGHTED AVERAGE ESTIMATES 



DO 3310 M=1,NPHASE 

Cl = (1. 96*EST(4,M))/SQRT(XNREP) 

CIU = EST(3,M) + Cl 
CIL = EST(3,M) - Cl 

WRITEC 30,3260) M,AREL(M) ,EST(3,M) ,EST(4,M) ,CIL,CIU 
3310 CONTINUE 

WRITEC 30, 3220) 

WRITEC 30, 3320) 

3320 FORMATC 'O' ,T43, 'REGRESSION ESTIMATE USING DISCOUNTED FAILURES') 
WRITEC 30, 3240) 

WRITEC 30, 3250) 

C COMPUTE C. I. FOR EXPONENTIAL REGRESSION ESTIMATES 

DO 3330 M=1,NPHASE 

Cl = Cl. 96*REGESTC4,M))/SQRTCXNREP) 
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CIU = REGEST(3,M) + Cl 
CIL = REGE3T(3,M) - Cl 

WRITE(30,3260) M,AREL(M) ,REGEST( 3 ,M) ,REGEST(4,M) ,CIL,CIU 
3330 CONTINUE 

WRITE(30,3340) 

3340 FORMAT( ' 1' ,T5 9, 'RECAPITULATION'//) 

WRITE(30,3350) 

3350 FORMATC ' ,T3, 'PHASE' , Til ,' ACTUAL' ,T28, 'MEAN' ,T38 'EST' ,T53, 'MEAN' 
C,T63,'EST' ,T78, 'MEAN' ,T88,'EST' ,T103, 'MEAN' ,T113, EST' ) 
WRITE(30,3360) 

3360 FORMATC ' ' ,T11 'RELIAB' ,T28,'WGT' ,T38, 'STD' ,T53,'MLE' ,T63, 'STD' ,T7 
C7, 'PHASE' ,T88/STD' ,T103, 'REG' ,T113, 'STD' ) 

WRITE(30,3370) 

3370 FORMATC ' ' ,T28,'AVG' ,T35 ,' DEVIATION' ,T53,'W/D' ,T60, 'DEVIATION' ,T78 
C, 'MLE' ,T85, 'deviation' ,T103, 'EST' ,T110 / DEVIATION ' ) 

WRITEC 30,3375) 

3375 FORMATC' ' ,T28 , ' EST' /) 

DO 650 I=1,NPHASE 

WRITEC 30,3380)1. ARELC I ) , ESTC 3 , 1 ) , ESTC 4,1), MLEWDC 3,1), MLEWDC 4,1), 
CMLESPC 3,1) ,MLESPC 4,1). REGESTC 3,1), REGESTC 4,1) 

3380 FORMATC 'o' ,T4 , 13 ,T11 ,F7. 6,T26,F7. 6,T36,F7. 6,T51,F7. 6,T61,F7. 6,T76, 
CF7. 6,T86,F7. 6,T101,F7. 6,T111,F7. 6) 

650 CONTINUE 
GO TO 6000 

4000 WRITEC 30, 40 10) 

4010 FORMATC IX,//, T50, 'FIXED PHASE RELIABILITY OPTION') 

WRITEC 30,4020) 

4020 FORMATC '-' ,T38, 'PHASE HUMBER' ,T78 ,' ACTUAL RELIABILITY') 

DO 4030 M=1,NPHASE 

WRITEC 30, 4040) M, ARELC M) 

4040 FORMATC 'o' ,T41, 12,T83 ,F8. 6) 

4030 CONTINUE 

GO TO 5000 
6000 CONTINUE 

STOP 

END 
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APPENDIX C. GRAPHICAL RESULTS 



This appendix contains the results of many of the various runs done with the sim- 
ulation. These results appear in tabular format in the output file but have been reduced 
to graphical form for ease of understanding. Below each graph of the estimates produced 
by the growth models are graphs of the standard deviations of each estimate by phase. 
This appendix is ordered by reliability growth pattern with Pattern I first and the re- 
mainder following sequentially. 
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Figure 27. Pattern I, No Discounting 
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Figure 28. Pattern I, F = .25, 1=1 
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Figure 29. Pattern I, F = .25, I = 3 
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Figure 30. Pattern I, F = .25, 1 = 6 
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Figure 31. Pattern I, F = .50, 1 = 3 
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Figure 32. Pattern I, F = .50, 1 = 6 
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Figure 33. Pattern I, F = .75, 1 = 3 
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Figure 34. Pattern I, F = .75, 1 = 6 
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Figure 35, Pattern I, Lloyd, Cl = .8 
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Figure 36. Pattern I, Lloyd, Cl = .9 
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Figure 37. Pattern II, No Discounting 
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Figure 38. Pattern II, F = .25, 1 = 3 
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Figure 39. Pattern II, F = .25, I = 6 
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Figure 40. Pattern II, F = .50, 1 = 3 
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Figure 41. Pattern II, F = .50, 1 = 6 
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Figure 42. Pattern II, F = .75, 1 = 3 
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Figure 43. Pattern II, F = .75, 1 = 6 
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Figure 44. Pattern II, Lloyd, Cl = .8 
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Figure 45. Pattern II, Lloyd, Cl = .9 
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Figure 46. Pattern III, No Discounting 
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Figure 47. Pattern III, F = .25, 1=1 
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Figure 48. Pattern III, F = .25, 1 = 3 
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Figure 49. Pattern III, F = .25, 1 = 6 
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Figure 50. Pattern III, F = .50, 1 = 3 
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Figure 51. Pattern III, F = .50, I = 6 
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Figure 52. Pattern III, F = .75, 1 = 3 
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Figure 53. Pattern III, F = .75, 1 = 6 
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Figure 54. Pattern III, Lloyd, Cl = .8 
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Figure 55. Pattern III, Lloyd, Cl = ,9 



102 



o 




PHASE 




PHASE 



Figure 56. Pattern IV, No Discounting 
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Figure 57. Pattern IV, F = .101 = 10 
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Figure 58. Pattern IV, F = .25, 1 = 25 
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Figure 59. Pattern IV, F = .25, 1 = 35 
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Figure 60. Pattern IV, F = .50, 1 = 60 
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Figure 61. Pattern IV, Lloyd, Cl = .8 
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Figure 62. Pattern IV, Lloyd, Cl = .9 
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Figure 63. Pattern IV, Lloyd, Cl = .99 
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Figure 6-1. Pattern V, No Discounting 



111 



q 




o Ll ! 1 1 1 ’l ilt 

2 4 5 8 10 

PHASE 




Figure 65. Pattern V, F = .25, 1=1 
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Figure 66. Pattern V, F = .25, 1 = 3 
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Figure 67. Pattern V, F = .25, 1 = 6 
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Figure 68. Pattern V, F = .50, 1 = 3 
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Figure 69. Pattern V, F = .50, 1 = 6 
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Figure 70. Pattern V, F = .75, 1 = 3 



117 



o 





PHASE 



Figure 71. Pattern V, F = .75, 1 = 6 
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Figure 72. Pattern V, Lloyd, Cl = .8 
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Figure 73. Pattern V, Lloyd, Cl = .9 
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Figure 74. Pattern VI, No Discounting 
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Figure 75. Pattern VI, F = .25, 1=1 
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Figure 76. Pattern VI, F = .25, 1 = 3 
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Figure 77. Pattern VI, F = .25, 1 = 6 
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Figure 78. Pattern VI, F = .50, 1 = 3 
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Figure 79. Pattern VI, F = .50, 1 = 6 
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Figure 80. Pattern VI, F = .75, 1 = 3 
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Figure 81. Pattern VI, F = .75, 1 = 6 
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Figure 82. Pattern VI, Lloyd, Cl = .8 
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Figure 83. Pattern VI, Lloyd, Cl = .9 
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Figure 84. Pattern VII, No Discounting 



131 



q 




ACTUAL RELIABILITY 
-Q- MLE SINGLE PHASE 
-V- EXP REG 
— ! — MLEFD 



2 4 6 a 10 




Figure 85. Pattern VII, F = .25, 1=1 
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Figure 86. Pattern VII, F = .25, 1 = 3 
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Figure 87. Pattern VII, F = .25, 1 = 6 
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Figure 88. Pattern VII, F = .50, 1 = 3 
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Figure 89. Pattern VII, F = .50, 1 = 6 
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Figure 90. Pattern VII, F = .75, 1 = 3 
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Figure 91. Pattern VII, F = .75, 1 = 6 
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Figure 92. Pattern VII, Lloyd, Cl = .8 
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Figure 93. Pattern VII, Lloyd, Cl = .9 
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Figure 94. Pattern VIII, No Discounting 
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Figure 95. Pattern VIII, F = .25, 1=1 
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Figure 96. Pattern VIII, F = .25, 1 = 3 
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Figure 97. Pattern VIII, F = .25, 1 = 6 
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Figure 98. Pattern Vlll, F = .50, 1 = 3 
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Figure 99. Pattern VIII, F = .50, 1 = 6 
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Figure 100. Pattern VIII, F — .75, 1 — 3 
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Figure 101. Pattern VIII, F = .75, 1 = 6 
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Figure 102. Pattern VIII, Lloyd, Cl = .8 
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Figure 103. Pattern VIII, Lloyd, Cl = .9 
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